From tjreedy at udel.edu Sun Jul 1 00:01:08 2012 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 30 Jun 2012 18:01:08 -0400 Subject: [Python-ideas] the optional "as" statement inside "if" statements In-Reply-To: References: Message-ID: On 6/30/2012 11:54 AM, Christopher Reay wrote: > The only hope for a large archive like this one is to wait long enough > to make sure you dont re hash the really regular ideas. > > ... ponders... Do I have time to read the archives? If you have time to post and discuss an idea, you have time to search gmane.comp.python.ideas at search.gmane.org and perhaps find out why not to bother posting. ? Do people mind adminiing the repetitive ideas? yes. -- Terry Jan Reedy From tjreedy at udel.edu Sun Jul 1 00:57:58 2012 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 30 Jun 2012 18:57:58 -0400 Subject: [Python-ideas] itertools.chunks(iterable, size, fill=None) In-Reply-To: References: Message-ID: <4FEF8476.1050609@udel.edu> On 6/29/2012 5:36 PM, Mike Graham wrote: > On Fri, Jun 29, 2012 at 4:32 PM, Georg Brandl wrote: >> so far there were no negative votes > > As far as I know, Raymond Hettinger is the itertools maintainer and he > has repeatedly objected to this idea in the past (e.g. > http://bugs.python.org/issue6021 ). Hopefully we can get his input > again. See my other post on this thread. I think people should really spend a few minutes researching before posting repeated ideas. In this case, read the itertools doc to see if such a function exists. It does, in the recipes, as 'grouper'. Has anyone ever before proposed that the recipe be added as a function? Search the tracker for 'itertools grouper'. Besides #6021, there are also http://bugs.python.org/issue1643 'Add group() to itertools' Raymond: "Sorry, I'm not interested in adding this to the module. Discussions to- date on the subject seem to show more interest in playing with grouper variants than in actual use cases. While the recipe given in the docs is somewhat opaque, it runs at C-speed (zero trips around the eval- loop) and it is encapsulated in a re-usable function. Writing this in C does nothing to improve the situation. Also, when people like to play with variants, there is no general agreement on useful requirements (like fill-in behavior or raising an exception on uneven length inputs). Trying to write option to meet all needs (n=2, step=1) makes the code more difficult to learn and use -- see several variants in Alex's Python Cookbook. Another issue is that we have to be very selective about adding tools to the module. Each addition makes the overall toolset harder to use -- it is better to have a good set of basic building blocks." http://bugs.python.org/issue13095 "Support for splitting lists/tuples into chunks" Raymond: "These have been rejected before. There is always a trade-off in adding tools such as this -- it can take more time to learn and remember them than to write a trivial piece of code to do it yourself. Another issue is that people tend to disagree on how to handle an odd sized left-over group -- different use cases require different handling. We're trying to keep the core toolset reasonably small so that python remains simple and learnable. That raises the threshold for adding new tools." There are now 18 itertools, up from the original. Grouper, or any generic function, may not be the best for what one wants with a list. I proposed in my other post that we *do* need a new doc section or how-to on this topic. (I am working on an outline.) -- Terry Jan Reedy From steve at pearwood.info Sun Jul 1 02:24:47 2012 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 01 Jul 2012 10:24:47 +1000 Subject: [Python-ideas] the optional "as" statement inside "if" statements In-Reply-To: References: Message-ID: <4FEF98CF.5050804@pearwood.info> Calvin Spealman wrote: > On Sat, Jun 30, 2012 at 9:59 AM, wrote: >> the idea is to make an variable assignment at the same time that the >> existence of that variable -- which is being returned by a function -- is >> made. >> >> suppose we are returning a variable from the method 'get' from the 'request' >> object and them making some stuff with it, but that stuff we will only do if >> it exists, if not, we'll just pass, instead of writing: >> >> variable = self.request.get('variable') >> if variable: >> print variable >> >> we could write >> >> if self.request.get('variable') as variable: >> print variable >> >> seems stupid (or not?), but with lots of variables to process, this >> pre-assignment could be very unpleasant -- especially if, as the in the >> example case, very little use will be made of the tested variable. >> >> also, the "as" expression already exists and is very pythonic. > > This is probably the best solution to the problem What problem? I'm not convinced that what fiatjaf at yahoo.com.br describes is an actual problem that needs solving. So what if you have to bind a reference to a name before using it in an if statement? That's not a problem. "Solving" it doesn't make programming any easier, or give you more power. There is absolutely no difference in programming power or expressiveness between: if func(a, b) as x: process(x) and x = func(a, b) if x: process(x) (In fact, I would argue that the second is *slightly* more readable, as the name binding comes first, not last.) At best, this proposal merely adds a trivial bit of syntactic sugar to the language. Contrast that with "import as", which truly does add expressiveness to the language. Working around the lack of "import as" if Python didn't have it would not be so easy. The obvious solution is also the wrong solution: # import long_name as ln import long_name ln = long_name del long_name That's wrong, because it has the side-effect of replacing, then deleting, any existing binding to long_name. "import as" does not do that. Good suggestions for syntactic sugar should add power or expressiveness to the language, not merely save a line of code or a few keystrokes. As given, this idea becomes one more special case for people to learn, with no corresponding benefit. -1 on the idea as given. By the way, to the Original Poster, please configure your mail client to display a name that we can refer to you by, or sign your posts. It doesn't have to be your real name, just something that you would like to be known as. It is annoying to have to refer to you by your email address. -- Steven From mwm at mired.org Sun Jul 1 02:48:52 2012 From: mwm at mired.org (Mike Meyer) Date: Sat, 30 Jun 2012 20:48:52 -0400 Subject: [Python-ideas] the optional "as" statement inside "if" statements In-Reply-To: References: Message-ID: <20120630204852.4b898a0a@bhuda.mired.org> On Sun, 1 Jul 2012 01:06:39 +1000 Nick Coghlan wrote: > This proposal has been considered and rejected many times. It's not > general enough - it *only* works for those cases where the value to be > retained *and* the interesting condition are the same. > > Consider the simple case of a value that may be either None (not > interesting) or a number (interesting). Since the interesting values > include "0", which evaluates as False along with None, this limited > form of embedded assignment syntax would not help. What Nick failed to point out is that the existing uses of "as" all bind *specific classes of objects*, not general ones. > Embedded assignment in C isn't that limited., but nobody has yet > volunteered to take the radical step of proposing "(X as Y)" as a > general embedded assignment syntax. I suggest anyone consider such an > idea do a *lot* of research in the python-ideas archives first, though > (as the idea has seen plenty of discussion). It is not as obviously > flawed as the if-and-while statement only variant, but it would still > involve being rather persuasive to make such a significant change to > the language. Hasn't the BDFL rejected a general embedded assignment in any case? As a final note, reading checking the archives for your ideas will also give you an idea of what kinds of things are and aren't accepted (i.e. - what's "Pythonic") as well as process - including the kinds of questions you'll be asked here - that an idea goes through before being accepted. http://www.mired.org/ Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From steve at pearwood.info Sun Jul 1 02:59:23 2012 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 01 Jul 2012 10:59:23 +1000 Subject: [Python-ideas] isascii()/islatin1()/isbmp() In-Reply-To: References: Message-ID: <4FEFA0EB.4030407@pearwood.info> Serhiy Storchaka wrote: > As shown in issue #15016 [1], there is a use cases when it is useful to > determine that string can be encoded in ASCII or Latin1. In working with > Tk or Windows console applications can be useful to determine that > string can be encoded in UCS2. C API provides interface for this, but at > Python level it is not available. > > I propose to add to strings class new methods: isascii(), islatin1() and > isbmp() (in addition to such methods as isalpha() or isdigit()). The > implementation will be trivial. > > Pro: The current trick with trying to encode has O(n) complexity and has > overhead of exception raising/catching. Are you suggesting that isascii and friends would be *better* than O(n)? How can that work -- wouldn't it have to scan the string and look at each character? Why just ASCII, Latin1 and BMP (whatever that is, googling has not come up with anything relevant)? It seems to me that adding these three tests will open the doors to a steady stream of requests for new methods is. I suggest that a better API would be a method that takes the name of an encoding (perhaps defaulting to 'ascii') and returns True|False: string.encodable(encoding='ascii') -> True|False Return True if string can be encoded using the named encoding, otherwise False. One last pedantic issue: strings aren't ASCII or Latin1, etc., but Unicode. There is enough confusion between Unicode text strings and bytes without adding methods whose names blur the distinction slightly. -- Steven From greg.ewing at canterbury.ac.nz Sun Jul 1 01:22:26 2012 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 01 Jul 2012 11:22:26 +1200 Subject: [Python-ideas] isascii()/islatin1()/isbmp() In-Reply-To: References: <20120630184316.5de2ce5e@pitrou.net> Message-ID: <4FEF8A32.50607@canterbury.ac.nz> Serhiy Storchaka wrote: > Several enchantments have already been rejected for this reason. ^^^^^^^^^^^^ Yeah, programming does seem to be a black art sometimes... -- Greg From tjreedy at udel.edu Sun Jul 1 04:05:21 2012 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 30 Jun 2012 22:05:21 -0400 Subject: [Python-ideas] isascii()/islatin1()/isbmp() In-Reply-To: <4FEFA0EB.4030407@pearwood.info> References: <4FEFA0EB.4030407@pearwood.info> Message-ID: On 6/30/2012 8:59 PM, Steven D'Aprano wrote: > Why just ASCII, Latin1 and BMP (whatever that is, googling has not come > up with anything relevant)? BMP = Unicode Basic MultilingualPlane, the first 2**16 codepoints http://unicode.org/roadmaps/bmp/ I presume the proposed isbmp would exclude surrogates in 16 bit implementations, but that was not clearly defined. > It seems to me that adding these three tests The temptation for these three tests is the the info is already available (at least for 2 of them) as an internal implementation-specific C-level attribute in 3.3(+). No O(n) scan needed. Someone could make a CPython3.3-specific module available on PyPI. > will open the doors to a steady stream of requests for new methods > is. > I suggest that a better API would be a method that takes the name of an > encoding (perhaps defaulting to 'ascii') and returns True|False: > > string.encodable(encoding='ascii') -> True|False > > Return True if string can be encoded using the named encoding, otherwise > False. But then one might as well try the encoding and check for exception. The point of the proposal is to avoid things like try: body = text.encode('ascii') header = 'ascii' #abbreviating here except UnicodeEncodeError: try: body = text.encode('latin1') header = 'latin1' except UnicodeEncodeError: body = text.encode('utf-8') header = 'utf-8' > One last pedantic issue: strings aren't ASCII or Latin1, etc., but > Unicode. There is enough confusion between Unicode text strings and > bytes without adding methods whose names blur the distinction slightly. yes! -- Terry Jan Reedy From steve at pearwood.info Sun Jul 1 05:21:32 2012 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 01 Jul 2012 13:21:32 +1000 Subject: [Python-ideas] isascii()/islatin1()/isbmp() In-Reply-To: References: <4FEFA0EB.4030407@pearwood.info> Message-ID: <4FEFC23C.3040805@pearwood.info> Terry Reedy wrote: > On 6/30/2012 8:59 PM, Steven D'Aprano wrote: >> I suggest that a better API would be a method that takes the name of an >> encoding (perhaps defaulting to 'ascii') and returns True|False: >> >> string.encodable(encoding='ascii') -> True|False >> >> Return True if string can be encoded using the named encoding, otherwise >> False. > > But then one might as well try the encoding and check for exception. The > point of the proposal is to avoid things like > > try: > body = text.encode('ascii') > header = 'ascii' #abbreviating here > except UnicodeEncodeError: > try: > body = text.encode('latin1') > header = 'latin1' > except UnicodeEncodeError: > body = text.encode('utf-8') > header = 'utf-8' Right. And re-written with the hypothetical encodable method, you have the usual advantage of LBYL that it is slightly more concise: body = header = None for encoding in ('ascii', 'latin1', 'utf-8'): if text.encodable(encoding): body = text.encode(encoding) header = encoding instead of: body = header = None for encoding in ('ascii', 'latin1', 'utf-8'): try: body = text.encode(encoding) header = encoding except UnicodeEncodeError: pass As for as expressibility goes, it is not much of an advantage. But: - if there are optimizations that apply to some encodings but not others, the encodable method can take advantage of them without it being a promise of the language; - it only adds a single string method (and presumably a single bytes method, decodable) rather than a plethora of methods; So, I don't care much either way for a LBYL test, but if there is a good use case for such a test, better for it to be a single method taking the encoding name rather than a multitude of tests, or exposing an implementation-specific value that the coder then has to interpret themselves. -1 on isascii, islatin1, isbmp -1 on exposing max_code_point +0.5 on encodable -- Steven From storchaka at gmail.com Sun Jul 1 07:31:05 2012 From: storchaka at gmail.com (Serhiy Storchaka) Date: Sun, 01 Jul 2012 08:31:05 +0300 Subject: [Python-ideas] isascii()/islatin1()/isbmp() In-Reply-To: <4FEF8A32.50607@canterbury.ac.nz> References: <20120630184316.5de2ce5e@pitrou.net> <4FEF8A32.50607@canterbury.ac.nz> Message-ID: On 01.07.12 02:22, Greg Ewing wrote: > Serhiy Storchaka wrote: >> Several enchantments have already been rejected for this reason. > ^^^^^^^^^^^^ > > Yeah, programming does seem to be a black art sometimes... Sorry, I mean enhancements. But yes, there is no good programming without magic. From stefan_ml at behnel.de Sun Jul 1 07:44:56 2012 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 01 Jul 2012 07:44:56 +0200 Subject: [Python-ideas] itertools.chunks(iterable, size, fill=None) In-Reply-To: <4FEF6B10.9040409@udel.edu> References: <4FEF6B10.9040409@udel.edu> Message-ID: Terry Reedy, 30.06.2012 23:09: > For the doc, I think it would be helpful here and in most module > subchapters if there were a subchapter table of contents at the top (under > 9.1 in this case). Even though just 2 lines here (currently, but see > below), it would let people know that there *is* a recipes section. After > the appropriate tables, mention that there are example uses in the recipe > section. Possibly add similar tables in the recipe section. +1, the recipes are interesting enough to be highly visible, both as usage examples and to solve actual problems. > Another addition could be a new subsection on grouping (chunking) that > would discuss post-processing of grouper (as discussed above), as well as > other recipes, including ones specific to strings and sequences. It would > essentially be a short how-to. Call it 9.1.3 "Grouping, Blocking, or > Chunking Sequences and Iterables". The synonyms will help external > searching. A toc would let people who have found this doc know to look for > this at the bottom. If it really is such an important use case for so many people, I agree that it's worth special casing it in the docs. It's not a trivial algorithmic step from a sequential iterable to a grouped iterable. Stefan From tjreedy at udel.edu Sun Jul 1 07:48:14 2012 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 01 Jul 2012 01:48:14 -0400 Subject: [Python-ideas] isascii()/islatin1()/isbmp() In-Reply-To: <4FEFC23C.3040805@pearwood.info> References: <4FEFA0EB.4030407@pearwood.info> <4FEFC23C.3040805@pearwood.info> Message-ID: On 6/30/2012 11:21 PM, Steven D'Aprano wrote: > Terry Reedy wrote: >> On 6/30/2012 8:59 PM, Steven D'Aprano wrote: > >>> I suggest that a better API would be a method that takes the name of an >>> encoding (perhaps defaulting to 'ascii') and returns True|False: >>> >>> string.encodable(encoding='ascii') -> True|False >>> >>> Return True if string can be encoded using the named encoding, otherwise >>> False. >> >> But then one might as well try the encoding and check for exception. >> The point of the proposal is to avoid things like >> >> try: >> body = text.encode('ascii') >> header = 'ascii' #abbreviating here >> except UnicodeEncodeError: >> try: >> body = text.encode('latin1') >> header = 'latin1' >> except UnicodeEncodeError: >> body = text.encode('utf-8') >> header = 'utf-8' > > Right. And re-written with the hypothetical encodable method, you have > the usual advantage of LBYL that it is slightly more concise: > > body = header = None > for encoding in ('ascii', 'latin1', 'utf-8'): > if text.encodable(encoding): > body = text.encode(encoding) > header = encoding But you are doing about half the work twice. > instead of: > > body = header = None > for encoding in ('ascii', 'latin1', 'utf-8'): > try: > body = text.encode(encoding) > header = encoding > except UnicodeEncodeError: > pass > As for as expressibility goes, it is not much of an advantage. But: > > - if there are optimizations that apply to some encodings but not others, > the encodable method can take advantage of them without it being a > promise of the language; It would be an optimization limited to a couple of encodings with CPython. Using it for cross-version code would be something like the trap of depending on the CPython optimization of repeated string concatenation. > - it only adds a single string method (and presumably a single bytes > method, decodable) rather than a plethora of methods; Decodable would always require a scan of the bytes. Might as well just decode and look for UnicodeDecodeError. > So, I don't care much either way for a LBYL test, but if there is a good > use case for such a test, My claim is that there is only a good use case if it is O(1), which would only be a few cases on CPython. > better for it to be a single method taking the > encoding name rather than a multitude of tests, or exposing an > implementation-specific value that the coder then has to interpret > themselves. > > -1 on isascii, islatin1, isbmp I do not see much of any use for isbmp. Maybe I missed something in the original post. > -1 on exposing max_code_point Jython and IronPython are stuck with the underlying platform implementations, which I believe are like the current semi-utf-16 narrow builds. So it would have to be a CPython-only attribute for now. (PyPy might consider adopting the new Unicode implementation someday too.) > +0.5 on encodable encodable would indirectly expose max_code_point since it would only be really useful and likely used when max_code_point was available and applicable. In other words, s.encodable('latin1') is equivalent to s.max_code_point == 255. if isbmp *is* useful, I don't think it can be duplicated with .encodable. Python seems not to have a ucs-2 codec. -- Terry Jan Reedy From ncoghlan at gmail.com Sun Jul 1 08:27:25 2012 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 1 Jul 2012 16:27:25 +1000 Subject: [Python-ideas] isascii()/islatin1()/isbmp() In-Reply-To: References: <4FEFA0EB.4030407@pearwood.info> <4FEFC23C.3040805@pearwood.info> Message-ID: On Sun, Jul 1, 2012 at 3:48 PM, Terry Reedy wrote: > encodable would indirectly expose max_code_point since it would only be > really useful and likely used when max_code_point was available and > applicable. In other words, s.encodable('latin1') is equivalent to > s.max_code_point == 255. > > if isbmp *is* useful, I don't think it can be duplicated with .encodable. > Python seems not to have a ucs-2 codec. Rewinding back to the reasons the question is being asked, the reason this information is useful at the Python level is the same reason it is useful at the C level: it matters for finding the most efficient means of representing the text as bytes (which can then have further implications for the kind of quoting used, etc). The interesting breakpoints can actually be expressed in terms of the number of bits in the highest code point: 7 - encode as ASCII (or latin-1 or utf-8) 8 - encode as latin-1 8+ - encode as utf-8 Specifically, it's a payload microoptimisation for the latin-1 case - the latin-1 string will be shorter than the corresponding utf-8 string (how much shorter depends on the number of non-ASCII characters). I believe it also makes an additional difference in the email case by changing the kind of quoting that is used to something with lower overhead that can't handle utf-8. The "try it and see" approach suffers a potentially high speed penalty if the non-latin-1 characters appear late in the string: try: # Likely no need to try ASCII, since there's no efficiency gain over latin-1 payload = message.encode("latin-1") except UnicodeEncodeError: payload = message.encode("utf-8") Using max() and ord() to check in advance doesn't help, since that *locks in* the O(n) penalty. The reason I think a max_code_point() method is a good potential solution is that it can be advertised as O(n) worst case, but potentially O(1) if the implementation caches the answer internally. Another alternative would be a __max__ and __min__ protocol that allowed efficient answers for the max() and min() builtins. The latter would have the advantage of allowing other containers (like range objects) to provide efficient implementations. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From raymond.hettinger at gmail.com Sun Jul 1 09:07:56 2012 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Sun, 1 Jul 2012 00:07:56 -0700 Subject: [Python-ideas] itertools.chunks(iterable, size, fill=None) In-Reply-To: References: <4FEF6B10.9040409@udel.edu> Message-ID: <2E44286F-D3DB-4739-89AF-3A8EAA9D8901@gmail.com> On Jun 30, 2012, at 10:44 PM, Stefan Behnel wrote: >> >> Another addition could be a new subsection on grouping (chunking) that >> would discuss post-processing of grouper (as discussed above), as well as >> other recipes, including ones specific to strings and sequences. It would >> essentially be a short how-to. Call it 9.1.3 "Grouping, Blocking, or >> Chunking Sequences and Iterables". The synonyms will help external >> searching. A toc would let people who have found this doc know to look for >> this at the bottom. > > If it really is such an important use case for so many people, I agree that > it's worth special casing it in the docs. It's not a trivial algorithmic > step from a sequential iterable to a grouped iterable. I'm not too keen on adding a section like this to the itertools docs. Instead, I would be open adding "further reading" section with external links to interesting iterator writeups in blogs, cookbooks, stack overflow answers, wikis, etc. If one of you wants to craft an elegant blog post on "Grouping, Blocking, or Chunking Sequences and Iterables", I would be happy to link to it. Raymond -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Sun Jul 1 18:22:33 2012 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 1 Jul 2012 18:22:33 +0200 Subject: [Python-ideas] Happy leap second References: Message-ID: <20120701182233.589552b5@pitrou.net> On Sat, 30 Jun 2012 10:23:45 -0400 Alexander Belopolsky wrote: > Even though many have hoped that the authorities would stop fiddling > with our clocks, today a leap second will be inserted in UTC. > Systems using Olson/IANA timezone database have a way to deal with > this without adjusting their clocks, but few systems are configured > that way: > > $ TZ=right/UTC date -d @1341100824 > Sat Jun 30 23:59:60 UTC 2012 If I believe the Internet, many server systems got broken by the leap second (*). Even one of our buildbots seems affected: - before: http://buildbot.python.org/all/builders/AMD64%20Ubuntu%20LTS%203.x/builds/419 - after: http://buildbot.python.org/all/builders/AMD64%20Ubuntu%20LTS%203.x/builds/420 (weird errors with timeouts, and a crash) (*) https://linuxfr.org/users/nono/journaux/leap-second Regards Antoine. From guido at python.org Sun Jul 1 21:11:25 2012 From: guido at python.org (Guido van Rossum) Date: Sun, 1 Jul 2012 21:11:25 +0200 Subject: [Python-ideas] Happy leap second In-Reply-To: <20120701182233.589552b5@pitrou.net> References: <20120701182233.589552b5@pitrou.net> Message-ID: So would "supporting" the leap second in Python have made any difference? On Sun, Jul 1, 2012 at 6:22 PM, Antoine Pitrou wrote: > On Sat, 30 Jun 2012 10:23:45 -0400 > Alexander Belopolsky > wrote: > >> Even though many have hoped that the authorities would stop fiddling >> with our clocks, today a leap second will be inserted in UTC. >> Systems using Olson/IANA timezone database have a way to deal with >> this without adjusting their clocks, but few systems are configured >> that way: >> >> $ TZ=right/UTC date -d @1341100824 >> Sat Jun 30 23:59:60 UTC 2012 > > If I believe the Internet, many server systems got broken by the leap > second (*). > Even one of our buildbots seems affected: > - before: > http://buildbot.python.org/all/builders/AMD64%20Ubuntu%20LTS%203.x/builds/419 > - after: > http://buildbot.python.org/all/builders/AMD64%20Ubuntu%20LTS%203.x/builds/420 > (weird errors with timeouts, and a crash) > > > (*) https://linuxfr.org/users/nono/journaux/leap-second > > Regards > > Antoine. > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas -- --Guido van Rossum (python.org/~guido) From solipsis at pitrou.net Sun Jul 1 21:43:35 2012 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 1 Jul 2012 21:43:35 +0200 Subject: [Python-ideas] Happy leap second References: <20120701182233.589552b5@pitrou.net> Message-ID: <20120701214335.59220b05@pitrou.net> On Sun, 1 Jul 2012 21:11:25 +0200 Guido van Rossum wrote: > So would "supporting" the leap second in Python have made any difference? No. These look like system (kernel?) bugs. Regards Antoine. From ncoghlan at gmail.com Mon Jul 2 00:14:03 2012 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 2 Jul 2012 08:14:03 +1000 Subject: [Python-ideas] Happy leap second In-Reply-To: <20120701214335.59220b05@pitrou.net> References: <20120701182233.589552b5@pitrou.net> <20120701214335.59220b05@pitrou.net> Message-ID: It was a kernel race condition (and subsequent live lock) after ntpd notified the clock subsystem of the coming leap second ( http://serverfault.com/questions/403732/anyone-else-experiencing-high-rates-of-linux-server-crashes-during-a-leap-second) The joys of fine grained locking :) Cheers, Nick. -- Sent from my phone, thus the relative brevity :) On Jul 2, 2012 5:45 AM, "Antoine Pitrou" wrote: > On Sun, 1 Jul 2012 21:11:25 +0200 > Guido van Rossum wrote: > > So would "supporting" the leap second in Python have made any difference? > > No. These look like system (kernel?) bugs. > > Regards > > Antoine. > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Mon Jul 2 10:38:19 2012 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 2 Jul 2012 18:38:19 +1000 Subject: [Python-ideas] isascii()/islatin1()/isbmp() In-Reply-To: References: <4FEFA0EB.4030407@pearwood.info> <4FEFC23C.3040805@pearwood.info> Message-ID: <20120702083819.GB27130@ando> On Sun, Jul 01, 2012 at 01:48:14AM -0400, Terry Reedy wrote: > >As for as expressibility goes, it is not much of an advantage. But: > > > >- if there are optimizations that apply to some encodings but not others, > > the encodable method can take advantage of them without it being a > > promise of the language; > > It would be an optimization limited to a couple of encodings with > CPython. Using it for cross-version code would be something like the > trap of depending on the CPython optimization of repeated string > concatenation. I'd hardly call it a trap. It's not like string concatenation which is expected to be O(N**2) on CPython but occasionally falls back to O(N**2). It would be O(N) expected on all platforms, but occasionally does better. Perhaps an anti-trap -- sometimes it does better than expected, rather than worse. > >- it only adds a single string method (and presumably a single bytes > > method, decodable) rather than a plethora of methods; > > Decodable would always require a scan of the bytes. Might as well just > decode and look for UnicodeDecodeError. *shrug* Perhaps so. bytes.decodable() would only be a LBYL convenience method. > >So, I don't care much either way for a LBYL test, but if there is a good > >use case for such a test, > > My claim is that there is only a good use case if it is O(1), which > would only be a few cases on CPython. *shrug* Again, I'm not exactly championing this proposal. I can see that an encodable method would be useful, but not that much more useful than trying to encode and catching the exception. A naive O(N) version of encodable() is trivial to implement. -- Steven From steve at pearwood.info Mon Jul 2 10:52:24 2012 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 2 Jul 2012 18:52:24 +1000 Subject: [Python-ideas] isascii()/islatin1()/isbmp() In-Reply-To: References: <4FEFA0EB.4030407@pearwood.info> <4FEFC23C.3040805@pearwood.info> Message-ID: <20120702085224.GC27130@ando> On Sun, Jul 01, 2012 at 04:27:25PM +1000, Nick Coghlan wrote: > Rewinding back to the reasons the question is being asked, the reason > this information is useful at the Python level is the same reason it > is useful at the C level: it matters for finding the most efficient > means of > representing the text as bytes (which can then have further > implications for the kind of quoting used, etc). The interesting > breakpoints can actually be expressed in terms of the number of bits > in the highest code point: > 7 - encode as ASCII (or latin-1 or utf-8) > 8 - encode as latin-1 > 8+ - encode as utf-8 I'm of two minds here. On the one hand, I question the wisdom of encouraging the use of anything but UTF-8. It's unfortunate enough that there are still cases where people have to use older encodings, without encouraging people to use Latin1 or ASCII in order to save a handful of bytes in a 20K email. On the other hand, there are use-cases for non-UTF-8 encodings, and people will want to check whether or not a string is encodable in various encodings. Why make that harder/slower/less convenient than it need be? > Specifically, it's a payload microoptimisation for the latin-1 case - > the latin-1 string will be shorter than the corresponding utf-8 string Just to be clear here, you're referring to byte strings, yes? > (how much shorter depends on the number of non-ASCII characters). I > believe it also makes an additional difference in the email case by > changing the kind of quoting that is used to something with lower > overhead that can't handle utf-8. > > The "try it and see" approach suffers a potentially high speed penalty > if the non-latin-1 characters appear late in the string: > > try: > # Likely no need to try ASCII, since there's no efficiency > gain over latin-1 > payload = message.encode("latin-1") > except UnicodeEncodeError: > payload = message.encode("utf-8") > > Using max() and ord() to check in advance doesn't help, since that > *locks in* the O(n) penalty. > > The reason I think a max_code_point() method is a good potential > solution is that it can be advertised as O(n) worst case, but > potentially O(1) if the implementation caches the answer internally. The downside is that the caller is then responsible for interpreting that value (i.e. mapping a max code point to an encoding). The other downside is that doesn't do anything to help those who are stuck with legacy encodings. Although maybe that doesn't matter, since they will just do the "try it and see" approach. > Another alternative would be a __max__ and __min__ protocol that > allowed efficient answers for the max() and min() builtins. The latter > would have the advantage of allowing other containers (like range > objects) to provide efficient implementations. +1 on that, although I think that should be a separate issue. -- Steven From ironfroggy at gmail.com Mon Jul 2 12:45:43 2012 From: ironfroggy at gmail.com (Calvin Spealman) Date: Mon, 2 Jul 2012 06:45:43 -0400 Subject: [Python-ideas] isascii()/islatin1()/isbmp() In-Reply-To: References: Message-ID: On Sat, Jun 30, 2012 at 12:03 PM, Serhiy Storchaka wrote: > As shown in issue #15016 [1], there is a use cases when it is useful to > determine that string can be encoded in ASCII or Latin1. In working with Tk > or Windows console applications can be useful to determine that string can > be encoded in UCS2. C API provides interface for this, but at Python level > it is not available. > > I propose to add to strings class new methods: isascii(), islatin1() and > isbmp() (in addition to such methods as isalpha() or isdigit()). The > implementation will be trivial. > > Pro: The current trick with trying to encode has O(n) complexity and has > overhead of exception raising/catching. > > Contra: In most cases after determining characters range we still need to > encode a string with the appropriate encoding. New methods will complicate > already overloaded strings class. > > Objections? -1 It doesn't make sense to special case them, instead of a simpler canencode() method added. It could save memory, but I don't see it saving time. > [1] http://bugs.python.org/issue15016 > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy From storchaka at gmail.com Mon Jul 2 13:32:50 2012 From: storchaka at gmail.com (Serhiy Storchaka) Date: Mon, 02 Jul 2012 14:32:50 +0300 Subject: [Python-ideas] isascii()/islatin1()/isbmp() In-Reply-To: <20120702085224.GC27130@ando> References: <4FEFA0EB.4030407@pearwood.info> <4FEFC23C.3040805@pearwood.info> <20120702085224.GC27130@ando> Message-ID: On 02.07.12 11:52, Steven D'Aprano wrote: >> Another alternative would be a __max__ and __min__ protocol that >> allowed efficient answers for the max() and min() builtins. The latter >> would have the advantage of allowing other containers (like range >> objects) to provide efficient implementations. > > +1 on that, although I think that should be a separate issue. This is issue #15226. From tjreedy at udel.edu Mon Jul 2 18:35:58 2012 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 02 Jul 2012 12:35:58 -0400 Subject: [Python-ideas] isascii()/islatin1()/isbmp() In-Reply-To: References: <4FEFA0EB.4030407@pearwood.info> <4FEFC23C.3040805@pearwood.info> <20120702085224.GC27130@ando> Message-ID: On 7/2/2012 7:32 AM, Serhiy Storchaka wrote: > On 02.07.12 11:52, Steven D'Aprano wrote: >>> Another alternative would be a __max__ and __min__ protocol that >>> allowed efficient answers for the max() and min() builtins. The latter >>> would have the advantage of allowing other containers (like range >>> objects) to provide efficient implementations. >> >> +1 on that, although I think that should be a separate issue. > > This is issue #15226. http://bugs.python.org/issue15226 was about exposing the max codepoint, which the OP thought was readily available in C, but is not. (It is at creation, but it is then replaced by 1 of 4 values.) -- Terry Jan Reedy From raymond.hettinger at gmail.com Tue Jul 3 06:23:29 2012 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Mon, 2 Jul 2012 21:23:29 -0700 Subject: [Python-ideas] [Python-Dev] itertools.chunks(iterable, size, fill=None) In-Reply-To: References: <4FEF6B10.9040409@udel.edu> <2E44286F-D3DB-4739-89AF-3A8EAA9D8901@gmail.com> Message-ID: <877C3C8D-1DE3-4DB0-812A-B352E8608364@gmail.com> On Jul 1, 2012, at 5:01 AM, Stefan Behnel wrote: > To address the main problem of users not finding what they need, what about > simply extending the docstring of the grouper() Here's a small change to the docstring: http://hg.python.org/cpython/rev/d32f21d87363 FWIW, if you're interested in load balancing applications, George Sakkis's itertools recipe for roundrobin() may be of interest. Another interesting iterator technique that is not well known is the two-argument form of iter() which is a marvel for transforming callables into iterators: for block in iter(partial(f.read, 1024), ''): ... for diceroll in iter(partial(randrange(1, 7), 4): ... Raymond -------------- next part -------------- An HTML attachment was scrubbed... URL: From techtonik at gmail.com Wed Jul 4 11:57:20 2012 From: techtonik at gmail.com (anatoly techtonik) Date: Wed, 4 Jul 2012 12:57:20 +0300 Subject: [Python-ideas] itertools.chunks(iterable, size, fill=None) In-Reply-To: References: Message-ID: On Fri, Jun 29, 2012 at 11:32 PM, Georg Brandl wrote: > On 26.06.2012 10:03, anatoly techtonik wrote: >> >> Now that Python 3 is all about iterators (which is a user killer >> feature for Python according to StackOverflow - >> http://stackoverflow.com/questions/tagged/python) would it be nice to >> introduce more first class functions to work with them? One function >> to be exact to split string into chunks. >> >> itertools.chunks(iterable, size, fill=None) >> >> Which is the 33th most voted Python question on SO - >> >> http://stackoverflow.com/questions/312443/how-do-you-split-a-list-into-evenly-sized-chunks-in-python/312464 >> >> P.S. CC'ing to python-dev@ to notify about the thread in python-ideas. >> > > Anatoly, so far there were no negative votes -- would you care to go > another step and propose a patch? Was about to say "no problem", but in fact - there is. Sorry from whining from my side and thanks for nudging. The only thought that a simple task of copy/pasting relevant code from http://docs.python.org/library/itertools.html?highlight=itertools#recipes will require a few hours waiting of download (still not everybody has a high-speed internet) makes me switch to other less time consuming tasks before getting around to it. These tasks become more important in a few hours, and basically I've passed through this many times before. It then becomes quite hard to switch back. I absolutely don't mind someone else being credited for the idea, because ideas usually worthless without implementation. It will be interesting to design how the process could work in a separate thread. For now the best thing I can do (I don't risk even to mention anything with 3.3) is to copy/paste code from the docs here: from itertools import izip_longest def chunks(iterable, size, fill=None): """Split an iterable into blocks of fixed-length""" # chunks('ABCDEFG', 3, 'x') --> ABC DEF Gxx args = [iter(iterable)] * size return izip_longest(fillvalue=fill, *args) BTW, this doesn't work as expected (at least for strings). Expected is: chunks('ABCDEFG', 3, 'x') --> 'ABC' 'DEF' 'Gxx' got: chunks('ABCDEFG', 3, 'x') --> ('A' 'B' 'C') ('D' 'E' 'F') ('G' 'x' 'x') Needs more round tuits definitely. From anacrolix at gmail.com Wed Jul 4 18:19:10 2012 From: anacrolix at gmail.com (Matt Joiner) Date: Thu, 5 Jul 2012 00:19:10 +0800 Subject: [Python-ideas] the optional "as" statement inside "if" statements In-Reply-To: <20120630204852.4b898a0a@bhuda.mired.org> References: <20120630204852.4b898a0a@bhuda.mired.org> Message-ID: There's no point doing this in Python since scoping is function-level and not lexical. Just move the assignment to the line preceding the if statement. On Sun, Jul 1, 2012 at 8:48 AM, Mike Meyer wrote: > On Sun, 1 Jul 2012 01:06:39 +1000 > Nick Coghlan wrote: >> This proposal has been considered and rejected many times. It's not >> general enough - it *only* works for those cases where the value to be >> retained *and* the interesting condition are the same. >> >> Consider the simple case of a value that may be either None (not >> interesting) or a number (interesting). Since the interesting values >> include "0", which evaluates as False along with None, this limited >> form of embedded assignment syntax would not help. > > What Nick failed to point out is that the existing uses of "as" all > bind *specific classes of objects*, not general ones. > >> Embedded assignment in C isn't that limited., but nobody has yet >> volunteered to take the radical step of proposing "(X as Y)" as a >> general embedded assignment syntax. I suggest anyone consider such an >> idea do a *lot* of research in the python-ideas archives first, though >> (as the idea has seen plenty of discussion). It is not as obviously >> flawed as the if-and-while statement only variant, but it would still >> involve being rather persuasive to make such a significant change to >> the language. > > Hasn't the BDFL rejected a general embedded assignment in any case? > > As a final note, reading checking the archives for your ideas will > also give you an idea of what kinds of things are and aren't accepted > (i.e. - what's "Pythonic") as well as process - including the kinds of > questions you'll be asked here - that an idea goes through before > being accepted. > > -- > Mike Meyer http://www.mired.org/ > Independent Software developer/SCM consultant, email for more information. > > O< ascii ribbon campaign - stop html mail - www.asciiribbon.org > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas From jeanpierreda at gmail.com Wed Jul 4 18:35:57 2012 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Wed, 4 Jul 2012 12:35:57 -0400 Subject: [Python-ideas] the optional "as" statement inside "if" statements In-Reply-To: References: <20120630204852.4b898a0a@bhuda.mired.org> Message-ID: On Wed, Jul 4, 2012 at 12:19 PM, Matt Joiner wrote: > There's no point doing this in Python since scoping is function-level > and not lexical. Just move the assignment to the line preceding the if > statement. I'm not sure what distinction you want to draw here between "lexical scoping" and "function-level scoping". Is your issue that scopes cannot be inferred during the lexing phase? Or is it something else? Generally, AIUI, because lexical scoping is only really interesting in contrast to static scoping, any variable whose scope can be determined statically based on where it's defined is considered "lexically scoped". -- Devin From masklinn at masklinn.net Wed Jul 4 19:25:01 2012 From: masklinn at masklinn.net (Masklinn) Date: Wed, 4 Jul 2012 19:25:01 +0200 Subject: [Python-ideas] the optional "as" statement inside "if" statements In-Reply-To: References: <20120630204852.4b898a0a@bhuda.mired.org> Message-ID: <2CDC2D81-0A20-4B93-B8BA-7BAD8C5D5C21@masklinn.net> On 2012-07-04, at 18:19 , Matt Joiner wrote: > There's no point doing this in Python since scoping is function-level > and not lexical. Just move the assignment to the line preceding the if > statement. Technically, scoping is *both* lexical and function-level (as opposed to block-based), although writing to a lexical scope is slightly more complex than in other languages with mutable lexical scopes. From masklinn at masklinn.net Wed Jul 4 19:30:12 2012 From: masklinn at masklinn.net (Masklinn) Date: Wed, 4 Jul 2012 19:30:12 +0200 Subject: [Python-ideas] the optional "as" statement inside "if" statements In-Reply-To: References: <20120630204852.4b898a0a@bhuda.mired.org> Message-ID: <09066A71-B626-4A35-8366-7FD3C9ED415C@masklinn.net> On 2012-07-04, at 18:35 , Devin Jeanpierre wrote: > On Wed, Jul 4, 2012 at 12:19 PM, Matt Joiner wrote: >> There's no point doing this in Python since scoping is function-level >> and not lexical. Just move the assignment to the line preceding the if >> statement. > > I'm not sure what distinction you want to draw here between "lexical > scoping" and "function-level scoping". Is your issue that scopes > cannot be inferred during the lexing phase? Or is it something else? Guessing he misused "lexical scoping" for block-based scoping (as in e.g. C) or explicit scoping constructs (e.g. `let`-type constructs). > Generally, AIUI, because lexical scoping is only really interesting in > contrast to static scoping And I see you miswrite as well, "lexical scoping" and "static scoping" are the same thing, and opposed to "dynamic scoping" (where name resolution traverses the call stack). From techtonik at gmail.com Thu Jul 5 15:36:24 2012 From: techtonik at gmail.com (anatoly techtonik) Date: Thu, 5 Jul 2012 16:36:24 +0300 Subject: [Python-ideas] itertools.chunks(iterable, size, fill=None) In-Reply-To: <4FEF6B10.9040409@udel.edu> References: <4FEF6B10.9040409@udel.edu> Message-ID: Before anything else I must apologize for significant lags in my replies. I can not read all of them to hold in my head, so I reply one by one as it goes trying not to miss a single point out there. It would be much easier to do this in unified interface for threaded discussions, but for now there is no capabilities for that neither in Mailman nor in GMail. And when it turns out that the amount of text is too big, and I spend a lot of time trying to squeeze it down and then it becomes pointless to send at all. Now back on the topic: On Sun, Jul 1, 2012 at 12:09 AM, Terry Reedy wrote: > On 6/29/2012 4:32 PM, Georg Brandl wrote: >> >> On 26.06.2012 10:03, anatoly techtonik wrote: >>> >>> Now that Python 3 is all about iterators (which is a user killer >>> feature for Python according to StackOverflow - >>> http://stackoverflow.com/questions/tagged/python) would it be nice to >>> introduce more first class functions to work with them? One function >>> to be exact to split string into chunks. > > Nothing special about strings. It seemed so, but it just appeared that grouper recipe didn't work for me. >>> itertools.chunks(iterable, size, fill=None) > > This is a renaming of itertools.grouper in 9.1.2. Itertools Recipes. You > should have mentioned this. I think of 'blocks' rather than 'chunks', but I > notice several SO questions with 'chunk(s)' in the title. I guess `block` gives too low signal/noize ration in search results. That's why it probably also called chunks in other languages, where `block` stand for something else (I speak of Ruby blocks). >>> Which is the 33th most voted Python question on SO - >>> >>> http://stackoverflow.com/questions/312443/how-do-you-split-a-list-into-evenly-sized-chunks-in-python/312464 > > I am curious how you get that number. I do note that there are about 15 > other Python SO questions that seem to be variations on the theme. There > might be more if 'blocks' and 'groups' were searched for. It's easy: 1. Go http://stackoverflow.com/ 2. Search [python] 3. Click `votes` tab 4. Choose `30 per page` at the bottom 5. Jump to the second page, there it is 4th from the top: http://stackoverflow.com/questions/tagged/python?page=2&sort=votes&pagesize=30 As for duplicates - feel free to mark them as such. SO allows everybody to do this (unlike Roundup). >> Anatoly, so far there were no negative votes -- would you care to go >> another step and propose a patch? > > That is because Raymond H. is not reading either list right now ;-) > Hence the Cc:. Also because I did not yet respond to a vague, very > incomplete idea. > > From Raymond's first message on http://bugs.python.org/issue6021 , add > grouper: > > "This has been rejected before. I quite often see such arguments and I can't stand to repeat that these are not arguments. It is good to know, but when people use that as a reason to close tickets - that's just disgusting. To the Raymond's honor he cares to explain. > * It is not a fundamental itertool primitive. The recipes section in > the docs shows a clean, fast implementation derived from zip_longest(). What is the definition of 'fundamental primitive'? To me the fact that top answer for chunking strings on SO has 2+ times more votes than itertools versions is a clear 5 sigma indicator that something is wrong with this Standard model without chunks boson. > * There is some debate on a correct API for odd lengths. Some people > want an exception, some want fill-in values, some want truncation, and > some want a partially filled-in tuple. The alone is reason enough not > to set one behavior in stone. use case 3.1: odd lengths exception (CHOOSE ONE) 1. I see that no itertools function throws exceptions, check manually: len(iterable) / float(size) == len(iterable) // float(size) 2. Explicitly - itertools.chunks(iterable, size, fill=None) + itertools.chunks(iterable, size, fill=None, exception=False) use case 3.2. fill in value. it is here (SOLVED) use case 3.3: truncation no itertools support truncation, do manually chunks(iter, size)[:len(iter)//size) use case 4: partially filled-in tuple What should be there? >>> chunks('ABCDEFG', 3, 'x') >>> | More replies and workarounds to some of the raised points are below. > * There is an issue with having too many itertools. The module taken as > a whole becomes more difficult to use as new tools are added." There can be only two reasons to that: * chosen basis is bad (many functions that are rarely used or easily emulated) * basis is good, but insufficient, because iterators universe is more complicated than we think > This is not to say that the question should not be re-considered. Given the > StackOverflow experience in addition to that of the tracker and python-list > (and maybe python-ideas), a special exception might be made in relation to > points 1 and 3. --[offtopic about Python enhancements / proposals feedback]-- Yes, without SO I probably wouldn't trigger this at all. Because tracker doesn't help with raising importance - there are no votes, no feature proposals, no "stars". And what I "like" the most is that very "nice" resolution status - "committed/rejected" - which doesn't say anything at all. Python list? I try not to disrupt the frequency there. Python ideas? Too low participation level for gathering signals. There are many people that read, support, but don't want to reply (don't want to stand out or just lazy). There are many outside who don't want to be subscribed at all. There are 2000+ people spending time on Python conferences all over the world each year we see only a couple of reactions for every Python idea here. Quite often there are mistakes and omissions that would be nice to correct and you can't. So StackOverflow really helps here, but it is a Q&A tool, which is still much better than ML that are solely for chatting, brainstorming and all the crazy reading / writing stuff. They don't help to develop ideas collaboratively. Quite often I am just lost in amount of text to handle. --[/offtopic]-- > It regard to point 2: many 'proposals', including Anatoly's, neglect this > detail. But the function has to do *something* when seqlen % grouplen != 0. > So an 'idea' is not really a concrete programmable proposal until > 'something' is specified. > > Exception -- not possible for an itertool until the end of the iteration > (see below). To raise immediately for sequences, one could wrap grouper. > > def exactgrouper(sequence, k): # untested > if len(sequence) % k: > raise ValueError('Sequence length {} must be a multiple of group length > {}'.format(len(sequence), k) > else: > return itertools.grouper(sequence, k) Right. Iterator is not a sequence, because it doesn't know the length of its sequence. The method should not belong to itertools at all then. Python 3 is definitely become more complicated. I'd prefer to keep separated from iterator stuff, but it seems more harder with every iteration. > Of course, sequences can also be directly sequentially sliced (but should > the result be an iterable or sequence of blocks?). But we do not have a > seqtools module and I do not think there should be another method added to > the seq protocol. I'd expect strings chunked into strings and lists into lists. Don't want to know anything about protocols. > Fill -- grouper always does this, with a default of None. > > Truncate, Remainder -- grouper (zip_longest) cannot directly do this and no > recipes are given in the itertools docs. (More could be, see below.) > > Discussions on python-list gives various implementations either for > sequences or iterables. For the latter, one approach is "it = > iter(iterable)" followed by repeated islice of the first n items. Another is > to use a sentinal for the 'fill' to detect a final incomplete block (tuple > for grouper). > > def grouper_x(n, iterable): # untested > sentinal = object() > for g in grouper(n, iterable, sentinal): > if g[-1] != sentinal: > yield g > else: > # pass to truncate > # yield g[:g.index(sentinal) for remainer > # raise ValueError for delayed exception We need a simple function to split a sequence into chunks(). Now we face with the problem to apply that technique to a sequence of infinite length when a last element of infinite sequence is encountered. You might be thinking now that this is a reduction to absurdity. But I'd say it is an exit from the trap. Mathematically this problem can't be solved. I am not ignoring your solution - I think it's quite feasible, but isn't it an overcomplication? I mean 160 people out of 149 who upvoted the question are pretty happy with an answer that just outputs the last chunk as-is: http://stackoverflow.com/questions/312443/how-do-you-split-a-list-into-evenly-sized-chunks-in-python chunks('ABCDEFG', 3) --> 'ABC' 'DEF' 'G' And it is quite nice solution to me, because you're free to do anything you'd like if you expect you data to be odd: for chunk in chunks('ABCDEFG', size): if len(chunk) < size: raise Tail You can make a helper iterator out of it too. > --- > The above discussion of point 2 touches on point 4, which Raymond neglected > in the particular message above but which has come up before: What are the > allowed input and output types? An idea is not a programmable proposal until > the domain, range, and mapping are specified. Domain? Mapping? I am not ignoring existing knowledge and experience. I just don't want to complicate and don't see appropriate `import usecase` in current context, so I won't try to guess what this means. in string -> out list of strings in list -> out list of lists > Possible inputs are a specific sequence (string, for instance), any > sequence, any iterable. Possible outputs are a sequence or iterator of > sequence or iterator. The various python-list and stackoverflow posts > questions asks for various combinations. zip_longest and hence grouper takes > any iterable and returns an iterator of tuples. (An iterator of maps might > be more useful as a building block.) This is not what one usually wants with > string input, for instance, nor with range input. To illustrate: Allright. Got it. Sequences have a length and can be sliced with [i:j], iterator can't be sliced (and hence no chunks can be made). So this function doesn't belong to itertools - it is a missing string or sequence method. We can't have a chunk with an iterator, because iterator over a string decomposes it into a group of pieces with no reverse function. We can have a group and then join the group into something. But this requires the knowledge of appropriate join() function for the iterator, and probably not efficient. As there are no such function (must be that Mapping you referenced above) - the recomposition into chunks is impossible. > import itertools as it > > def grouper(n, iterable, fillvalue=None): > "grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx" > args = [iter(iterable)] * n > return it.zip_longest(*args, fillvalue=fillvalue) > > print(*(grouper(3, 'ABCDEFG', 'x'))) # probably not wanted > print(*(''.join(g) for g in grouper(3, 'ABCDEFG', 'x'))) > # > ('A', 'B', 'C') ('D', 'E', 'F') ('G', 'x', 'x') > ABC DEF Gxx > > -- > What to do? One could easily write 20 different functions. So more thought > is needed before adding anything. -1 on the idea as is. I've learned a new English type of argument - "straw man" (I used to call this "hijacking"). This -1 doesn't belong to original idea. It belongs to proposal of itertools.chunks() with a long list of above points and completely different user stories (i.e. not "split string into chunks"). I hope you still +1 with 160 people on SO that think Python needs an easy way to chunk sequences. > For the doc, I think it would be helpful here and in most module subchapters > if there were a subchapter table of contents at the top (under 9.1 in this > case). Even though just 2 lines here (currently, but see below), it would > let people know that there *is* a recipes section. After the appropriate > tables, mention that there are example uses in the recipe section. Possibly > add similar tables in the recipe section. Unfortunately, it appeared that grouper() is not chunks(). It doesn't delivers list of list of chars given string as an input instead of list of chunks. > Another addition could be a new subsection on grouping (chunking) that would > discuss post-processing of grouper (as discussed above), as well as other > recipes, including ones specific to strings and sequences. It would > essentially be a short how-to. Call it 9.1.3 "Grouping, Blocking, or > Chunking Sequences and Iterables". The synonyms will help external > searching. A toc would let people who have found this doc know to look for > this at the bottom. This makes matters pretty ugly. In ideal language there should be less docs, not more. From techtonik at gmail.com Thu Jul 5 15:47:53 2012 From: techtonik at gmail.com (anatoly techtonik) Date: Thu, 5 Jul 2012 16:47:53 +0300 Subject: [Python-ideas] [Python-Dev] itertools.chunks(iterable, size, fill=None) In-Reply-To: References: <4FEF6B10.9040409@udel.edu> <2E44286F-D3DB-4739-89AF-3A8EAA9D8901@gmail.com> Message-ID: On Sun, Jul 1, 2012 at 3:01 PM, Stefan Behnel wrote: > > To address the main problem of users not finding what they need, what about > simply extending the docstring of the grouper() function with a sentence > like this: > > "This functionality is also called 'chunking' or 'blocking' and can be used > for load distribution and sharding." > > That would make it easy for users to find what they are looking for when > they search the page for "chunk". I find that a much more common and less > ambiguous name than "grouping", which reminds me more of "group by". In appeared that "chunking" and "grouping" are different kind of tasks. You can chunk a sequence (sting) by slicing it into smaller sequences, but you can not chunk in iterable - you can only group it. There is an data loss about the structure that occurs when a sequence (string) becomes an iterator: chunks ABCDE -> AB CD E group ABCDE -> A B C D E -> (A B) (C D) (D E) From techtonik at gmail.com Thu Jul 5 16:33:19 2012 From: techtonik at gmail.com (anatoly techtonik) Date: Thu, 5 Jul 2012 17:33:19 +0300 Subject: [Python-ideas] [Python-Dev] itertools.chunks(iterable, size, fill=None) In-Reply-To: References: Message-ID: On Wed, Jul 4, 2012 at 9:31 PM, Terry Reedy wrote: > On 7/4/2012 5:57 AM, anatoly techtonik wrote: >> >> On Fri, Jun 29, 2012 at 11:32 PM, Georg Brandl wrote: > > >>> Anatoly, so far there were no negative votes -- would you care to go >>> another step and propose a patch? >> >> >> Was about to say "no problem", > > > Did you read that there *are* strong negative votes? And that this idea has > been rejected before? I summarized the objections in my two responses and > pointed to the tracker issues. One of the objections is that there are 4 > different things one might want if the sequence length is not an even > multiple of the chunk size. Your original 'idea' did not specify. I actually meant that there is a problem to propose a patch in the sense of getting checkout, working on a diff, sending it by attaching to bug tracker as developer guide says. >> For now the best thing I can do (I don't risk even to mention anything >> with 3.3) is to copy/paste code from the docs here: >> >> from itertools import izip_longest >> def chunks(iterable, size, fill=None): >> """Split an iterable into blocks of fixed-length""" >> # chunks('ABCDEFG', 3, 'x') --> ABC DEF Gxx >> args = [iter(iterable)] * size >> return izip_longest(fillvalue=fill, *args) > > > Python ideas is about Python 3 ideas. Please post Python 3 code. > > This is actually a one liner > > return zip_longest(*[iter(iterable)]*size, fillvalue=file) > > We don't generally add such to the stdlib. Can you figure out from the code what this stuff does? It doesn't give chunks of strings. >> BTW, this doesn't work as expected (at least for strings). Expected is: >> chunks('ABCDEFG', 3, 'x') --> 'ABC' 'DEF' 'Gxx' >> got: >> chunks('ABCDEFG', 3, 'x') --> ('A' 'B' 'C') ('D' 'E' 'F') ('G' 'x' 'x') > > > One of the problems with idea of 'add a chunker' is that there are at least > a dozen variants that different people want. That's not the problem. People always want something extra. The problem that we don't have a real wish distribution. If 1000 people want chunks and 1 wants groups with exception - we still account these as equal variants. Therefore my idea is deliberately limited to "string to chunks" user story, and SO implementation proposal. > I discussed the problem of > return types issue in my responses. I showed how to get the 'expected' > response above using grouper, but also suggested that it is the wrong basis > for splitting strings. Repeated slicing make more sense for concrete > sequence types. > > def seqchunk_odd(s, size): > # include odd size left over > for i in range(0, len(s), size): > yield s[i:i+size] > > print(list(seqchunk_odd('ABCDEFG', 3))) > # > ['ABC', 'DEF', 'G'] Right. That's the top answer on SO that people think should be in stdlib. Great we are talking about the same thing actually. > def seqchunk_even(s, size): > # only include even chunks > for i in range(0, size*(len(s)//size), size): > yield s[i:i+size] > > print(list(seqchunk_even('ABCDEFG', 3))) > # > ['ABC', 'DEF'] This is deducible from seqchunk_odd(s, size) > def strchunk_fill(s, size, fill): > # fill odd chunks > q, r = divmod(len(s), size) > even = size * q > for i in range(0, even, size): > yield s[i:i+size] > if size != even: > yield s[even:] + fill * (size - r) > > print(list(strchunk_fill('ABCDEFG', 3, 'x'))) > # > ['ABC', 'DEF', 'Gxx'] Also deducible from seqchunk_odd(s, size) > Because the 'fill' value is necessarily a sequence for strings, > strchunk_fill would only work for lists and tuples if the fill value were > either required to be given as a tuple or list of length 1 or if it were > internally converted inside the function. Skipping that for now. > > Having written the fill version based on the even version, it is easy to > select among the three behaviors by modifying the fill version. > > def strchunk(s, size, fill=NotImplemented): > # fill odd chunks > q, r = divmod(len(s), size) > even = size * q > for i in range(0, even, size): > yield s[i:i+size] > if size != even and fill is not NotImplemented: > yield s[even:] + fill * (size - r) > > print(*strchunk('ABCDEFG', 3)) > print(*strchunk('ABCDEFG', 3, '')) > print(*strchunk('ABCDEFG', 3, 'x')) > # > ABC DEF > ABC DEF G > ABC DEF Gxx I now don't even think that fill value is needed as argument. if len(chunk) < size: chunk.extend( [fill] * ( size - len(chunk)) ) > I already described how something similar could be done by checking each > grouper output tuple for a fill value, but that requires that the fill value > be a sentinal that could not otherwise appear in the tuple. One could modify > grouper to fill with a private object() and check the last item of each > group for that sentinal and act accordingly (delete, truncate, or replace). > A generic api needs some thought, though. I just need to chunk strings and sequences. Generic API is too complex without counting all usecases and iterating over them. > An issue I did not previously mention is that people sometimes want > overlapping chunks rather than contiguous disjoint chunks. The slice > approach trivially adapts to that. > > def seqlap(s, size): > for i in range(len(s)-size+1): > yield s[i:i+size] > > print(*seqlap('ABCDEFG', 3)) > # > ABC BCD CDE DEF EFG > > A sliding window for a generic iterable requires a deque or ring buffer > approach that is quite different from the zip-longest -- grouper approach. That's why I'd like to drastically reduce the scope of proposal. itertools doesn't seem to be the best place anymore. How about sequence method? string.chunks(size) -> ABC DEF G list.chunks(size) -> [A,B,C], [C,D,E],[G] If somebody needs a keyword argument - this can come later without breaking compatibility. From steve at pearwood.info Thu Jul 5 17:57:17 2012 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 06 Jul 2012 01:57:17 +1000 Subject: [Python-ideas] [Python-Dev] itertools.chunks(iterable, size, fill=None) In-Reply-To: References: Message-ID: <4FF5B95D.3060208@pearwood.info> anatoly techtonik wrote: > On Wed, Jul 4, 2012 at 9:31 PM, Terry Reedy wrote: >> A sliding window for a generic iterable requires a deque or ring buffer >> approach that is quite different from the zip-longest -- grouper approach. > > That's why I'd like to drastically reduce the scope of proposal. > itertools doesn't seem to be the best place anymore. How about > sequence method? > > string.chunks(size) -> ABC DEF G > list.chunks(size) -> [A,B,C], [C,D,E],[G] -1 This is a fairly trivial problem to solve, and there are many variations on it. Many people will not find the default behaviour helpful, and will need to write their own. Why complicate the API for all sequence types with this? I don't believe that we should enshrine one variation as a built-in method, without any evidence that it is the most useful or common variation. Even if there is one variation far more useful than the others, that doesn't necessarily mean we ought to make it a builtin method unless it is a fundamental sequence operation, has wide applicability, and is genuinely hard to write. I don't believe chunking meets *any* of those criteria, let alone all three. Not every six line function needs to be a builtin. I believe that splitting a sequence (or a string) into fixed-size chunks is more of a programming exercise problem than a genuinely useful tool. That does not mean that there is never any real use-cases for splitting into fixed-size chunks, only that this is the function that *seems* more useful in theory than it turns out in practice. Compare this with more useful sequence/iteration tools, like (say) zip. You can hardly write a hundred lines of code without using zip at least once. But I bet you can write tens of thousands of lines of code without needing to split sequences into fixed chunks like this. Besides, the name "chunks" is more general than how you are using it. For example, I consider chunking to be splitting a sequence up at a various delimiters or separators, not at fixed character positions. E.g. "the third word of item two of the fourth line" is a chunk. This fits more with the non-programming use of the term chunk or chunking, and has precedence in Apple's Hypertalk language, which literally allowed you to talk about words, items and lines of text, each of which are described as chunks. This might be a good candidate for a utility module made up of assorted useful functions, but not for the string and sequence APIs. -- Steven From steve at pearwood.info Thu Jul 5 18:09:44 2012 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 06 Jul 2012 02:09:44 +1000 Subject: [Python-ideas] [Python-Dev] itertools.chunks(iterable, size, fill=None) In-Reply-To: References: <4FEF6B10.9040409@udel.edu> Message-ID: <4FF5BC48.9080302@pearwood.info> anatoly techtonik wrote: >>>> Which is the 33th most voted Python question on SO - >>>> >>>> http://stackoverflow.com/questions/312443/how-do-you-split-a-list-into-evenly-sized-chunks-in-python/312464 >> I am curious how you get that number. I do note that there are about 15 >> other Python SO questions that seem to be variations on the theme. There >> might be more if 'blocks' and 'groups' were searched for. > > It's easy: > 1. Go http://stackoverflow.com/ > 2. Search [python] > 3. Click `votes` tab > 4. Choose `30 per page` at the bottom > 5. Jump to the second page, there it is 4th from the top: > http://stackoverflow.com/questions/tagged/python?page=2&sort=votes&pagesize=30 Yes. I don't think this is particularly significant. Have a look at some of the questions with roughly the same number of votes: #26 "How can I remove (chomp) a newline in Python?" 176 votes #33 "How do you split a list into evenly sized chunks in Python?" 149 votes #36 "Accessing the index in Python for loops" 144 votes Being 33rd most voted question doesn't really mean much. By the way, why is this discussion going to both python-dev and python-ideas? -- Steven From techtonik at gmail.com Thu Jul 5 21:41:29 2012 From: techtonik at gmail.com (anatoly techtonik) Date: Thu, 5 Jul 2012 22:41:29 +0300 Subject: [Python-ideas] Bloody FAQ (Was: [Python-Dev] itertools.chunks(iterable, size, fill=None)) Message-ID: On Thu, Jul 5, 2012 at 7:50 PM, Stefan Behnel wrote: > anatoly techtonik, 05.07.2012 15:36: >> On Sun, Jul 1, 2012 at 12:09 AM, Terry Reedy wrote: >>> From Raymond's first message on http://bugs.python.org/issue6021 , add >>> grouper: >>> >>> "This has been rejected before. >> >> I quite often see such arguments and I can't stand to repeat that >> these are not arguments. It is good to know, but when people use that >> as a reason to close tickets - that's just disgusting. > > The *real* problem is that people keep bringing up topics (and even spell > them out in the bug tracker) without searching for existing discussions > and/or tickets first. That's why those who do such a search (or who know > what they are talking about anyway) close these tickets with the remark > "this has been rejected before", instead of repeating an entire heap of > arguments all over again to feed a discussion that would only lead to the > same result as it did before, often several times before. Make the bloody FAQ and summarize this stuff? Why waste each others time? If people don't enjoy repeating themselves over and over - there is a bloody wiki. What should happen to people to start extracting gems of knowledge from piles of dusty sheets called list "archives" for others to admire. No, it is easier to say "it was already discussed many times", "why don't you Google yourself", "so far you're only complaining", etc. If people can't find anything - why everybody thinks they are ignorant and lazy. Even if it so, why nobody thinks that maybe that bloody Xapian index is dead again for a bloody amount of moons nobody knows why and how many exactly? Why nobody thinks that lazy coders can also help with development? Maybe that laziness is the primary reason some major groups actually prefer Python to Java, C++ and other more interesting languages (such as PHP) when it comes to typing? Make it easy and the patches will follow. Answers like "this was discussed before" don't make it easy to understand, and leaving users rereading old 19xx archives that people don't reread themselves will likely make users bounce and never (NEVER!) come up with some proposal again. An "organic" way to keep traffic low. Miscommunication is a bad experience for users, bad experience for developers, everybody is annoyed and as a result such nice language as Python loses points on TIOBE (and convenient chunk() functions to munch-munch on the sequence data). Wheew. :-F From techtonik at gmail.com Thu Jul 5 21:55:26 2012 From: techtonik at gmail.com (anatoly techtonik) Date: Thu, 5 Jul 2012 22:55:26 +0300 Subject: [Python-ideas] Python as a tool to download stuff for bootstrapping Message-ID: This one is practical. I am looking at NaCl SDK download page: https://developers.google.com/native-client/sdk/download "you need Python installed", "download SDK update utility" What makes me sad that update utility is a Python script in a zip file - nacl_sdk.zip which includes shell script and a .bat file for launching this Python script. This makes me kind of sad. You have Python installed. Why can't you just crossplatformly do: mkdir nacl cd nacl python -m urllib get http://commondatastorage.googleapis.com/nativeclient-mirror/nacl/nacl_sdk/update_sdk.py python update_sdk.py From amauryfa at gmail.com Thu Jul 5 22:24:27 2012 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Thu, 5 Jul 2012 22:24:27 +0200 Subject: [Python-ideas] Python as a tool to download stuff for bootstrapping In-Reply-To: References: Message-ID: 2012/7/5 anatoly techtonik : > This makes me kind of sad. You have Python installed. Why can't you > just crossplatformly do: > > mkdir nacl > cd nacl > python -m urllib get > http://commondatastorage.googleapis.com/nativeclient-mirror/nacl/nacl_sdk/update_sdk.py > python update_sdk.py I'm sure there is already a way with standard python tools. Something along these lines: python -c "from urllib.request import urlretrieve; urlretrieve('URL', 'update_sdk.zip')" python -m update_sdk.zip The second command will work if the zip file has a __main__.py. Do you think we need other tools? -- Amaury Forgeot d'Arc From simon.sapin at kozea.fr Thu Jul 5 22:15:10 2012 From: simon.sapin at kozea.fr (Simon Sapin) Date: Thu, 05 Jul 2012 22:15:10 +0200 Subject: [Python-ideas] Python as a tool to download stuff for bootstrapping In-Reply-To: References: Message-ID: <4FF5F5CE.8020701@kozea.fr> Le 05/07/2012 21:55, anatoly techtonik a ?crit : > This one is practical. I am looking at NaCl SDK download page: > https://developers.google.com/native-client/sdk/download > > "you need Python installed", "download SDK update utility" > > What makes me sad that update utility is a Python script in a zip file > - nacl_sdk.zip > which includes shell script and a .bat file for launching this Python script. > > This makes me kind of sad. You have Python installed. Why can't you > just crossplatformly do: > > mkdir nacl > cd nacl > python -m urllib get > http://commondatastorage.googleapis.com/nativeclient-mirror/nacl/nacl_sdk/update_sdk.py > python update_sdk.py Hi, "Sadness" drama aside, what?s your point? Is this a proposal to add a command-line API for urlretrieve()? In that case, +1. (For what my vote is worth.) The use cases are limited (just use the Python API) but they do exist, as you showed. Regards, -- Simon Sapin From ned at nedbatchelder.com Thu Jul 5 23:07:14 2012 From: ned at nedbatchelder.com (Ned Batchelder) Date: Thu, 05 Jul 2012 17:07:14 -0400 Subject: [Python-ideas] Bloody FAQ (Was: [Python-Dev] itertools.chunks(iterable, size, fill=None)) In-Reply-To: References: Message-ID: <4FF60202.1040909@nedbatchelder.com> On 7/5/2012 3:41 PM, anatoly techtonik wrote: > Make the bloody FAQ and summarize this stuff? Why waste each others > time? If people don't enjoy repeating themselves over and over - there > is a bloody wiki. What should happen to people to start extracting > gems of knowledge from piles of dusty sheets called list "archives" > for others to admire. > > No, it is easier to say "it was already discussed many times", "why > don't you Google yourself", "so far you're only complaining", etc. If > people can't find anything - why everybody thinks they are ignorant > and lazy. Even if it so, why nobody thinks that maybe that bloody > Xapian index is dead again for a bloody amount of moons nobody knows > why and how many exactly? Why nobody thinks that lazy coders can also > help with development? Maybe that laziness is the primary reason some > major groups actually prefer Python to Java, C++ and other more > interesting languages (such as PHP) when it comes to typing? Make it > easy and the patches will follow. Answers like "this was discussed > before" don't make it easy to understand, and leaving users rereading > old 19xx archives that people don't reread themselves will likely make > users bounce and never (NEVER!) come up with some proposal again. An > "organic" way to keep traffic low. > > Miscommunication is a bad experience for users, bad experience for > developers, everybody is annoyed and as a result such nice language as > Python loses points on TIOBE (and convenient chunk() functions to > munch-munch on the sequence data). Anatoly, miscommunication is a bad experience, yes. This is an excellent example of it. I honestly have no idea what you are advocating in this message, partly because sarcasm doesn't work well. You seem really upset, but I for one don't know what it is you want. --Ned. > Wheew. :-F > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From wuwei23 at gmail.com Fri Jul 6 07:05:04 2012 From: wuwei23 at gmail.com (alex23) Date: Thu, 5 Jul 2012 22:05:04 -0700 (PDT) Subject: [Python-ideas] Bloody FAQ (Was: [Python-Dev] itertools.chunks(iterable, size, fill=None)) In-Reply-To: References: Message-ID: <05347f1b-17b9-48e5-a99b-c744b4c8e258@nw7g2000pbb.googlegroups.com> On Jul 6, 5:41?am, anatoly techtonik wrote: > Why nobody thinks that lazy coders can also > help with development? Maybe that laziness is the primary reason some > major groups actually prefer Python to Java, C++ and other more > interesting languages (such as PHP) when it comes to typing? You make this criticism fairly regularly and yet you yourself constantly make suggestions and refuse to provide implementations. I have a hard time reconciling the implicit anger in this statement: > Why waste each others time? If people don't enjoy repeating themselves over > and over - there is a bloody wiki. ...with ones you have previously made defending your _own_ unwillingness to examine pre-existing discussion: > It's too boring to live in a world of existing knowledge and expertise You yourself feel you have no responsibility in understanding a situation before demanding that it be raised for discussion. I'm not sure _what_ others could put into place that would _make_ you perform this action. > Answers like "this was discussed > before" don't make it easy to understand, and leaving users rereading > old 19xx archives that people don't reread themselves will likely make > users bounce and never (NEVER!) come up with some proposal again. Once again you're putting the burden on _others_ to somehow come up with mechanisms to force people to do the due diligence they should be capable of doing as a programmer. Frankly, I'm of the opinion that if someone cannot be bothered putting the effort into understanding the pre-existing content for a proposal they're making, then yes, they simply should not come up with that proposal. I'm not entirely sure why you feel people should be entitled to request far more than they're willing to contribute. From wuwei23 at gmail.com Fri Jul 6 07:08:09 2012 From: wuwei23 at gmail.com (alex23) Date: Thu, 5 Jul 2012 22:08:09 -0700 (PDT) Subject: [Python-ideas] Bloody FAQ (Was: [Python-Dev] itertools.chunks(iterable, size, fill=None)) In-Reply-To: <05347f1b-17b9-48e5-a99b-c744b4c8e258@nw7g2000pbb.googlegroups.com> References: <05347f1b-17b9-48e5-a99b-c744b4c8e258@nw7g2000pbb.googlegroups.com> Message-ID: <55df23be-bfb5-40f7-aa51-6b7e89c129e3@m2g2000pbv.googlegroups.com> On Jul 6, 3:05?pm, alex23 wrote: > the pre-existing content That should actually be "context". From stephen at xemacs.org Fri Jul 6 07:26:57 2012 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 06 Jul 2012 14:26:57 +0900 Subject: [Python-ideas] itertools.chunks(iterable, size, fill=None) In-Reply-To: References: <4FEF6B10.9040409@udel.edu> Message-ID: <87fw95793y.fsf@uwakimon.sk.tsukuba.ac.jp> Annoying cross-post trimmed. anatoly techtonik writes: > Before anything else I must apologize for significant lags in my > replies. No, that's nothing you need to apologize for. Taking time to formulate a decent reply is common courtesy, and commendable. We're really *not* in a hurry here; Python is going to be around for a few more decades at least. What you need to apologize for is the major faux pas of cross-posting. Please cut it out, and in particular trim your own address list when replying, or use Reply-To (and maybe Mail-Followup-To) to redirect others' replies to an appropriate list. IMO, the right list for this discussion is python-ideas, but at the very least choose one. and again he writes: > Make the bloody FAQ and summarize this stuff? Why waste each others > time? If people don't enjoy repeating themselves over and over - > there is a bloody wiki. What should happen to people to start > extracting gems of knowledge from piles of dusty sheets called list > "archives" for others to admire. This is a terrible idea for python-dev and python-ideas. While it is frustrating to get a "been there, done that, rejected with extreme prejudice" reply, and there's no question that searching the archives is a hit-and-mostly-miss kind of thing because of the difficulty of choosing good search terms, it's really not that costly to come back with "I'm sorry, I couldn't find the thread". Rather than spend effort on writing a FAQ that would rather quickly turn into a monstrosity hardly more easy to search than the archives themselves, and almost never be read, we should devote any effort to improving the capability for searching archives (and the wiki and the issue tracker). The problem with trying to put everything into the FAQ is that it's a terribly unrewarding thing to do. Most of the stuff you will write will be ignored and profit nobody. It really needs to be selected, by somebody with taste and knowledge of user needs. Presumably O'Reilly or somebody has a "Python Hacks" book -- go buy it, which will encourage the author to keep up the good work. If they don't, why don't you propose it to them, and write it yourself? Then you can measure how good an idea it is by your royalties. Or if you don't feel like writing it yourself, maybe you can convince O'Reilly to come up with a big enough advance to interest somebody like Terry Reedy or Steven d'Aprano or even Raymond Hettinger or David Beazley. Regards, From mwm at mired.org Fri Jul 6 08:00:30 2012 From: mwm at mired.org (Mike Meyer) Date: Fri, 06 Jul 2012 02:00:30 -0400 Subject: [Python-ideas] itertools.chunks(iterable, size, fill=None) In-Reply-To: <87fw95793y.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4FEF6B10.9040409@udel.edu> <87fw95793y.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <1a7c74ab-084e-4c87-ade1-27d8c7f0b788@email.android.com> "Stephen J. Turnbull" wrote: >anatoly techtonik writes: > > Make the bloody FAQ and summarize this stuff? Why waste each others > > time? If people don't enjoy repeating themselves over and over - > > there is a bloody wiki. What should happen to people to start > > extracting gems of knowledge from piles of dusty sheets called list > > "archives" for others to admire. > >This is a terrible idea for python-dev and python-ideas. While it is >frustrating to get a "been there, done that, rejected with extreme >prejudice" reply, and there's no question that searching the archives >is a hit-and-mostly-miss kind of thing because of the difficulty of >choosing good search terms, it's really not that costly to come back >with "I'm sorry, I couldn't find the thread". Rather than spend >effort on writing a FAQ that would rather quickly turn into a >monstrosity hardly more easy to search than the archives themselves, >and almost never be read, we should devote any effort to improving the >capability for searching archives (and the wiki and the issue >tracker). Python-dev i'm not familiar enough with to formulate an opinion, but I think it could work well for python-ideas. While better search tools always help, a mail list is mostly unorganized, and a wiki need a purpose (possibly provided by a person or small group of people who take on the responsibility of maintaining the thing), or it becomes an even less organized mess. A FAQ, on the other hand, *has* a purpose. It also has a natural source of new material in the list. Python.org already has multiple FAQs with an integrated search engine (that are hopefully integrated into the python.org search engine), and last time I looked (admitadly long ago), a tool for maintaining FAQs. While I don't think I'm quaified to write answer for python-ideas faqs, I believe I can decide whether or not something is appropriate as a python-ideas FAQ entry. If the tool for maintinaing them isn't a memory error and is still in use, I'd certainly be willing to do that. But that's the easy job. The trick is getting the list members to play along. If they provide what is most people consider the definitive answer a couple of times, then the next time the question comes up, write it up as a FAQ entry and submit it as such as well to the list. The time after that, just provide a pointer to the FAQ entry. This worked very well for the FreeBSD FAQ, which is a fairly large document. They didn't even have a spiffy tool for maintaining it, but had to take submissions and add or fix markup and then check them in to the document tree. -- Sent from my Android tablet. Please excuse my swyping. From stephen at xemacs.org Fri Jul 6 11:22:03 2012 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 06 Jul 2012 18:22:03 +0900 Subject: [Python-ideas] itertools.chunks(iterable, size, fill=None) In-Reply-To: <1a7c74ab-084e-4c87-ade1-27d8c7f0b788@email.android.com> References: <4FEF6B10.9040409@udel.edu> <87fw95793y.fsf@uwakimon.sk.tsukuba.ac.jp> <1a7c74ab-084e-4c87-ade1-27d8c7f0b788@email.android.com> Message-ID: <8762a16y84.fsf@uwakimon.sk.tsukuba.ac.jp> Mike Meyer writes: > Python-dev i'm not familiar enough with to formulate an opinion, > but I think it could work well for python-ideas. You're missing the point. Improving the FAQ for python-*list* is a great idea, but Mr. techtonik proposes a FAQ for python-*ideas*. IOW, if the question is actually on-topic, the answers are reasons why something isn't going to ever be in Python. True, often the explanation of why a proposed feature is inappropriate is of the form "this three-line function does the job", but if the question is really a FAQ, it presumably has been asked multiple times on python-list, and should be picked up there. > This worked very well for the FreeBSD FAQ, which is a fairly large > document. No, it didn't. The FreeBSD FAQ is quite obviously not oriented to telling developers what isn't going to go into the next version of FreeBSD and why not. If people want to move ahead with this discussion, it really ought to move to python-list IMO. From mwm at mired.org Fri Jul 6 17:38:42 2012 From: mwm at mired.org (Mike Meyer) Date: Fri, 6 Jul 2012 11:38:42 -0400 Subject: [Python-ideas] itertools.chunks(iterable, size, fill=None) In-Reply-To: <8762a16y84.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4FEF6B10.9040409@udel.edu> <87fw95793y.fsf@uwakimon.sk.tsukuba.ac.jp> <1a7c74ab-084e-4c87-ade1-27d8c7f0b788@email.android.com> <8762a16y84.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20120706113842.28e7a3ae@bhuda.mired.org> On Fri, 06 Jul 2012 18:22:03 +0900 "Stephen J. Turnbull" wrote: > Mike Meyer writes: > > Python-dev i'm not familiar enough with to formulate an opinion, > > but I think it could work well for python-ideas. > You're missing the point. Improving the FAQ for python-*list* is a > great idea, but Mr. techtonik proposes a FAQ for python-*ideas*. IOW, > if the question is actually on-topic, the answers are reasons why > something isn't going to ever be in Python. That point I got. > True, often the explanation of why a proposed feature is inappropriate > is of the form "this three-line function does the job", but if the > question is really a FAQ, it presumably has been asked multiple times > on python-list, and should be picked up there. That I didn't, and it's fair enough. But the onus to fix this *still* falls on the people who are answering the same question over and over. They need to start submitting the answers to the FAQ (there is a history & design FAQ that would seem to be an appropriate place for them) rather than just to the list. > > This worked very well for the FreeBSD FAQ, which is a fairly large > > document. > No, it didn't. The FreeBSD FAQ is quite obviously not oriented to > telling developers what isn't going to go into the next version of > FreeBSD and why not. If you're claiming that the FreeBSD FAQ technique didn't work well for it, then I disagree. That the FreeBSD FAQ has a different orientation is true, but not clearly relevant. > If people want to move ahead with this discussion, it really ought to > move to python-list IMO. Not in mine. The problem is repeated questions on -ideas. My proposed solution has to be implemented by -ideas readers. They need to start submitting frequently typed answers to the appropriate FAQ, not just to the list. http://www.mired.org/ Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From jae+python at jaerhard.com Fri Jul 6 20:14:04 2012 From: jae+python at jaerhard.com (=?iso-8859-1?B?SvxyZ2VuIEEu?= Erhard) Date: Fri, 6 Jul 2012 20:14:04 +0200 Subject: [Python-ideas] itertools recipes: why not add them to the stdlib *somewhere*? Message-ID: <20120706181404.GB3347@jaerhard.com> I just now ran into (since learning about the recipes) having use for a recipe (then I found out that I can't use it anyway, but that's a different thing) So... I consider it very strange and annoying that there's this code. Ready to use. But you have to copy-and-paste it into your code *somewhere* instead of easily importing it. As to the "it makes using Python harder to learn", I beg to differ: an addition to the *language* (like, say, metaclasses) *can* make it more complicated. But additions to the stdlib? What about "batteries included"? Not a motto anymore (I heard rumors, so maybe that's the case) Putting these in some official(!) iterutils (or name it what you want) package would be a solution for so many people. Yes, not everyone needs grouper's current functionality. But for the many who do, it'd be there already. And someone thought those recipes are useful, didn't you? So why not make them more easily available? And then there's Alex Martelli at http://stackoverflow.com/a/1641242/238406, and I agree with him wholeheartedly. Bye, J From stephen at xemacs.org Fri Jul 6 20:53:31 2012 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 07 Jul 2012 03:53:31 +0900 Subject: [Python-ideas] itertools.chunks(iterable, size, fill=None) In-Reply-To: <20120706113842.28e7a3ae@bhuda.mired.org> References: <4FEF6B10.9040409@udel.edu> <87fw95793y.fsf@uwakimon.sk.tsukuba.ac.jp> <1a7c74ab-084e-4c87-ade1-27d8c7f0b788@email.android.com> <8762a16y84.fsf@uwakimon.sk.tsukuba.ac.jp> <20120706113842.28e7a3ae@bhuda.mired.org> Message-ID: <87txxk67ro.fsf@uwakimon.sk.tsukuba.ac.jp> Mike Meyer writes: > That I didn't, and it's fair enough. But the onus to fix this *still* > falls on the people who are answering the same question over and > over. They need to start submitting the answers to the FAQ (there is a > history & design FAQ that would seem to be an appropriate place for > them) rather than just to the list. I disagree that it's appropriate. The History & Design FAQ isn't intended to be encyclopedic, and I don't see that making it so would be useful. The point is that if "This has been rejected before" is sufficient answer, the proponent withdraws, everybody happy. And if not, they have to go to the archives anyway. They need to answer the defects brought up in the list discussion, point by point, or they're not going to get a hearing. > If you're claiming that the FreeBSD FAQ technique didn't work well > for it, then I disagree. No, I'm claiming that AFAICS the FreeBSD FAQ didn't even try to fix the problem that you and anatoly percieve. > The problem is repeated questions on -ideas. Indeed! But I think the solution is to encourage asking those questions on python-list first.[1] The experts there are both willing to answer such questions, and have a better sense of FAQ-iness (relatively few users interested in the question will propose a general solution). And they have an incentive to document answers if they notice a FAQ. If no suitable idiom appears, then work up a change proposal and post to -ideas. Footnotes: [1] Yes, I'm being mildly ironic; people typically won't know it's a FAQ until they ask. Nevertheless, asking on python-list before posting to either python-ideas or python-dev should be encouraged ("unless you're Dutch"). From g.brandl at gmx.net Fri Jul 6 21:30:41 2012 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 06 Jul 2012 21:30:41 +0200 Subject: [Python-ideas] Python as a tool to download stuff for bootstrapping In-Reply-To: References: Message-ID: On 05.07.2012 22:24, Amaury Forgeot d'Arc wrote: > 2012/7/5 anatoly techtonik : >> This makes me kind of sad. You have Python installed. Why can't you >> just crossplatformly do: >> >> mkdir nacl >> cd nacl >> python -m urllib get >> http://commondatastorage.googleapis.com/nativeclient-mirror/nacl/nacl_sdk/update_sdk.py >> python update_sdk.py > > I'm sure there is already a way with standard python tools. Something > along these lines: > > python -c "from urllib.request import urlretrieve; urlretrieve('URL', > 'update_sdk.zip')" > python -m update_sdk.zip > > The second command will work if the zip file has a __main__.py. > Do you think we need other tools? The "python -m urllib" (don't think "get" is required) interface certainly looks nice and is similar in style with many of the other __main__ stuff we add to stdlib modules. Georg From anacrolix at gmail.com Sat Jul 7 18:03:15 2012 From: anacrolix at gmail.com (Matt Joiner) Date: Sun, 8 Jul 2012 00:03:15 +0800 Subject: [Python-ideas] the optional "as" statement inside "if" statements In-Reply-To: <09066A71-B626-4A35-8366-7FD3C9ED415C@masklinn.net> References: <20120630204852.4b898a0a@bhuda.mired.org> <09066A71-B626-4A35-8366-7FD3C9ED415C@masklinn.net> Message-ID: Sorry yes I meant block scoping. I assimilated terms from further up in the thread I think. I mean that there can be no temporary/block/localized scope for a name to within the if statement, so there's no reason not to put it on the line above. FWIW I've grown fond of function level scoping, it really reduces confusion in closures and encourages smaller functions to avoid name leaks. Go has something like the OP asks for: if x := SomeFunc(); x > 3 { // x scoped only within this block } Haskell also lets you name expressions like: someFunc y at x:xs = ... But in Python I still think this is best: x = some_func() if x > 3: # no scoping :) On Thu, Jul 5, 2012 at 1:30 AM, Masklinn wrote: > On 2012-07-04, at 18:35 , Devin Jeanpierre wrote: > >> On Wed, Jul 4, 2012 at 12:19 PM, Matt Joiner wrote: >>> There's no point doing this in Python since scoping is function-level >>> and not lexical. Just move the assignment to the line preceding the if >>> statement. >> >> I'm not sure what distinction you want to draw here between "lexical >> scoping" and "function-level scoping". Is your issue that scopes >> cannot be inferred during the lexing phase? Or is it something else? > > Guessing he misused "lexical scoping" for block-based scoping (as in > e.g. C) or explicit scoping constructs (e.g. `let`-type constructs). > >> Generally, AIUI, because lexical scoping is only really interesting in >> contrast to static scoping > > And I see you miswrite as well, "lexical scoping" and "static scoping" > are the same thing, and opposed to "dynamic scoping" (where name > resolution traverses the call stack). > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas From masklinn at masklinn.net Sat Jul 7 18:15:35 2012 From: masklinn at masklinn.net (Masklinn) Date: Sat, 7 Jul 2012 18:15:35 +0200 Subject: [Python-ideas] the optional "as" statement inside "if" statements In-Reply-To: References: <20120630204852.4b898a0a@bhuda.mired.org> <09066A71-B626-4A35-8366-7FD3C9ED415C@masklinn.net> Message-ID: <24E42BCE-2875-4DFB-9E60-1B8A1CF9A7AA@masklinn.net> On 2012-07-07, at 18:03 , Matt Joiner wrote: > > Haskell also lets you name expressions like: > > someFunc y at x:xs = ... as-patterns have very little relation with what was described in the thread, as they're about keeping the "packed" version of a pattern-matched expression. Guards are probably closer: case someFunc val of x | x > 3 -> ? From christopherreay at gmail.com Sat Jul 7 20:07:02 2012 From: christopherreay at gmail.com (Christopher Reay) Date: Sat, 7 Jul 2012 19:07:02 +0100 Subject: [Python-ideas] Explanation on how to search the archives Message-ID: I have heard two suggestions on how to search the achives one is: If you have time to post and discuss an idea, you have time to search gmane.comp.python.ideas at search.gmane.org and perhaps find out why not to bother posting. And the Other: It's more a matter of working out how to point Google (or the search engine of your choice) at the archives in a useful way. In this case: https://www.google.com/search?q=inurl%3Apython-ideas%20site%3Amail.python.org%20embedded%20assignment This is great information. Is there a page suggesting how to search archives effectively? Whilst much of that skill is generic, and clear concise explanation of what is expected from the polite user of the mailing list (and other Python mailing lists) could help. A link to that page could be put in the footer of the python mailing list emails.. and voila, a very easy thing to reference and get righteous about people not having read. What do you think? Christopher -- Be prepared to have your predictions come true -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Sat Jul 7 21:01:59 2012 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 08 Jul 2012 05:01:59 +1000 Subject: [Python-ideas] Explanation on how to search the archives In-Reply-To: References: Message-ID: <4FF887A7.5070101@pearwood.info> Christopher Reay wrote: > Is there a page suggesting how to search archives effectively? [...] > What do you think? I think that is an excellent idea for some person or persons to write up on their blogs or other websites. There's no need to wait for permission, or for The One True Official Page -- anyone can write up a page and link to it. There is even a python.org wiki, sadly under-utilized in my opinion. Anyone with an account who wants to start a page there can. Not every concept vaguely related to programming in Python falls under the responsibility of python.org. Teaching people how to search the Internet and interact with mailing lists without being annoying is not the job of the Python devs, but it's a good way for people to help out. -- Steven From mikegraham at gmail.com Sun Jul 8 22:22:43 2012 From: mikegraham at gmail.com (Mike Graham) Date: Sun, 8 Jul 2012 16:22:43 -0400 Subject: [Python-ideas] Syntax for defining parametric decorators Message-ID: A common stumbling block for new users is writing decorators that take arguments. To create a decorator like @timesn(n) def f(y): ... We write code like def timesn(n) def decorator(f): def inner(y): return n * f(y) return inner return decorator which confuses many users and can be a handful to type. I wonder if it would be clearer for people to write def timesn(n)(f): def inner(y): return n * f(y) return inner which is more concise and looks a lot more like a non-parametric decorator someone might have written already. The syntax is mostly self-explaining and could potentially be useful in other contexts. There exist tools like the decorator library to try to simplify this already, but in my experience they mostly serve to confuse people using decorators for the first time more. One thing I didn't specify was whether `n` was nonlocal or not and the behavior of something that keeps and reuses timesn(some_specific_n) multiple times. Does anyone think a feature like this may be useful? Regards, Mike From jeanpierreda at gmail.com Sun Jul 8 22:41:02 2012 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Sun, 8 Jul 2012 16:41:02 -0400 Subject: [Python-ideas] Syntax for defining parametric decorators In-Reply-To: References: Message-ID: On Sun, Jul 8, 2012 at 4:22 PM, Mike Graham wrote: > which is more concise and looks a lot more like a non-parametric > decorator someone might have written already. The syntax is mostly > self-explaining and could potentially be useful in other contexts. Ooh, +1 Semantically the function and the decorator-params are both "arguments" to the decorator, and this confuses a lot of people into writing silly things like def decorator(arg1, arg2, f): ... In actuality, these two are equivalent, and this new def syntax reflects that: @decorator(arg): def foo(...): ... def foo(...): ... foo = decorator(arg)(foo) As opposed to the usual definition syntax, which is not really intuitive. > One thing I didn't specify was whether `n` was nonlocal or not and the > behavior of something that keeps and reuses timesn(some_specific_n) > multiple times. For the purposes of decorators I don't think it matters. I would guess that keeping and reusing the first arg is more useful for non-decorator purposes, and making them a nonlocal is more consistent with the rest of Python, if we reuse the argument. Plus it'd align better semantically with the old way of defining parametric decorators. -- Devin From solipsis at pitrou.net Sun Jul 8 23:52:46 2012 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 8 Jul 2012 23:52:46 +0200 Subject: [Python-ideas] Syntax for defining parametric decorators References: Message-ID: <20120708235246.7eaa54e8@pitrou.net> On Sun, 8 Jul 2012 16:22:43 -0400 Mike Graham wrote: > > which confuses many users and can be a handful to type. I wonder if it > would be clearer for people to write > > def timesn(n)(f): > def inner(y): > return n * f(y) > return inner But then, why not: def timesn(n)(f)(y): return n * f(y) ? Regards Antoine. -- Software development and contracting: http://pro.pitrou.net From ncoghlan at gmail.com Mon Jul 9 00:54:07 2012 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 9 Jul 2012 08:54:07 +1000 Subject: [Python-ideas] Syntax for defining parametric decorators In-Reply-To: References: Message-ID: Step one is to stop referring to decorator factories as decorators. Beyond that, no, this is too limited - it only helps when there's no extra code in the outer scope which, in my experience, is the exception rather than the rule. Closures are an advanced programming concept - there are limits to how simple they are ever going to be. Cheers, Nick. -- Sent from my phone, thus the relative brevity :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From grosser.meister.morti at gmx.net Mon Jul 9 01:13:40 2012 From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=) Date: Mon, 09 Jul 2012 01:13:40 +0200 Subject: [Python-ideas] Syntax for defining parametric decorators In-Reply-To: References: Message-ID: <4FFA1424.30009@gmx.net> On 07/08/2012 10:22 PM, Mike Graham wrote: > A common stumbling block for new users is writing decorators that take > arguments. To create a decorator like > > @timesn(n) > def f(y): > ... > > We write code like > > def timesn(n) > def decorator(f): > def inner(y): > return n * f(y) > return inner > return decorator > > which confuses many users and can be a handful to type. I wonder if it > would be clearer for people to write > > def timesn(n)(f): > def inner(y): > return n * f(y) > return inner > Why not write this?: def timesn(n)(f)(y): return n * f(y) This would be a currified function. One could implement something like that like this: def curry(f): for i in range(f.func_code.co_argcount-1): f = (lambda f: lambda *args: partial(f,*args))(f) return f @curry def timesn(n,f,y): return n * f(y) > which is more concise and looks a lot more like a non-parametric > decorator someone might have written already. The syntax is mostly > self-explaining and could potentially be useful in other contexts. > > There exist tools like the decorator library to try to simplify this > already, but in my experience they mostly serve to confuse people > using decorators for the first time more. > > One thing I didn't specify was whether `n` was nonlocal or not and the > behavior of something that keeps and reuses timesn(some_specific_n) > multiple times. > > Does anyone think a feature like this may be useful? > > Regards, > Mike From jeanpierreda at gmail.com Mon Jul 9 01:47:07 2012 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Sun, 8 Jul 2012 19:47:07 -0400 Subject: [Python-ideas] Syntax for defining parametric decorators In-Reply-To: <4FFA1424.30009@gmx.net> References: <4FFA1424.30009@gmx.net> Message-ID: On Sun, Jul 8, 2012 at 7:13 PM, Mathias Panzenb?ck wrote: > This would be a currified function. One could implement something like that > like this: > > def curry(f): > for i in range(f.func_code.co_argcount-1): > f = (lambda f: lambda *args: partial(f,*args))(f) > return f > > @curry > def timesn(n,f,y): > return n * f(y) But then how do you do: def linearcomposition(m, c=0)(f)(x): return m * f(x) + c Python isn't Haskell, so we can't really currify every argument and expect things to work out, esp. because of variadic functions (defaults or even *args and **kwargs). -- Devin From steve at pearwood.info Mon Jul 9 02:52:15 2012 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 09 Jul 2012 10:52:15 +1000 Subject: [Python-ideas] Syntax for defining parametric decorators In-Reply-To: References: Message-ID: <4FFA2B3F.5010106@pearwood.info> Mike Graham wrote: > A common stumbling block for new users is writing decorators that take > arguments. To create a decorator like I question that assertion. Have you spent much time on the tutor at python.org mailing list? I have, and I can say that decorators is not something that many new users there are concerned about *at all*, let alone decorators which take arguments. I believe that decorator factories, and closures in general, are a moderately advanced technique, and not a stumbling block for most new users. I can't say I remember the last time I've seen anyone ask for help writing decorator factories on either the tutor or python-list mailing list. So I don't believe that making new syntax is neither needed or desirable. -1 on the idea. Some more issues/problems with the idea: > We write code like > > def timesn(n) > def decorator(f): > def inner(y): > return n * f(y) > return inner > return decorator In my experience, most decorator factories (or factory-factories in general, not just for generators but any time you need to wrap a factory function in a closure) require at least one pre-processing step, and occasionally one or two post-processing steps as well: def timesn(n) pre_process() def decorator(f): pre_process() @functools.wraps(f) def inner(y): return n * f(y) post_process() return inner post_process() return decorator With your suggested syntax, you lose a scope, and it is unclear to me what this would do: def timesn(n)(f): pre_process() @functools.wraps(f) def inner(f): return n * f(y) post_process() return inner > which confuses many users and can be a handful to type. I wonder if it > would be clearer for people to write > > def timesn(n)(f): > def inner(y): > return n * f(y) > return inner > > which is more concise and looks a lot more like a non-parametric > decorator someone might have written already. The syntax is mostly > self-explaining and could potentially be useful in other contexts. Concise, yes, but I disagree that it is self-explaining. I'm having a lot of difficulty in thinking of how I would explain it to even a moderately experienced user except by expanding it out to the explicit nested function form. I'm not sure I can even explain it to myself. What will happen when the user invariably writes something like this? def ordinary(a)(b): return a + b My guess is that they will get a syntax error, but it will be a syntax error in the wrong place: instead of clearly flagging the error (a)(b) as invalid syntax, as you get now py> def ordinary(a)(b): File "", line 1 def ordinary(a)(b): ^ SyntaxError: invalid syntax Instead, the user will get a less useful syntax error on the following line, where the compiler sees that the function def is not followed immediately by another function def. py> def ordinary(a)(b): ... return a + b File "", line 1 return a + b ^ SyntaxError: invalid syntax So whatever clarity you (allegedly) gain when writing decorator factories, you lose when making an error. > There exist tools like the decorator library to try to simplify this > already, but in my experience they mostly serve to confuse people > using decorators for the first time more. In my opinion, that's because the decorator and decorator factory concept are already as simple as they can possibly be. Anything you do to them adds complexity, not reduces it. I mean, it's a simple concept: a decorator is a kind of function which returns a function. If you put that inside a third function, you have a function which returns a function which returns a function. Any syntax to disguise that simplicity is only going to make things more complex, not less. I haven't given any thought to how this will effect class decorators, but I suspect it will run into the same problems. -- Steven From mikegraham at gmail.com Mon Jul 9 03:11:00 2012 From: mikegraham at gmail.com (Mike Graham) Date: Sun, 8 Jul 2012 21:11:00 -0400 Subject: [Python-ideas] Syntax for defining parametric decorators In-Reply-To: References: Message-ID: On Sun, Jul 8, 2012 at 6:54 PM, Nick Coghlan wrote: > Beyond that, no, this is too limited - it only helps when there's no extra > code in the outer scope which, in my experience, is the exception rather > than the rule. I'm not sure I think that's the case. functools.wraps, functools.lru_cache, and reprlib.recursive_repr are the only stdlib decorator factories I could think of, and all of them could use this syntax. So could the ones I could quickly find in my own code. Mike From mikegraham at gmail.com Mon Jul 9 03:27:51 2012 From: mikegraham at gmail.com (Mike Graham) Date: Sun, 8 Jul 2012 21:27:51 -0400 Subject: [Python-ideas] Syntax for defining parametric decorators In-Reply-To: <4FFA2B3F.5010106@pearwood.info> References: <4FFA2B3F.5010106@pearwood.info> Message-ID: On Sun, Jul 8, 2012 at 8:52 PM, Steven D'Aprano wrote: > Mike Graham wrote: >> >> A common stumbling block for new users is writing decorators that take >> arguments. To create a decorator like > > I question that assertion. Have you spent much time on the tutor at python.org > mailing list? I have, and I can say that decorators is not something that > many new users there are concerned about *at all*, let alone decorators > which take arguments. I haven't, but I have spend considerable time helping people learn Python. I'm a gold-badge Python answerer on Stack Overflow, an op/regular on the #python IRC channel on freenode, and have taught people in the meatosphere. Decorators is certainly a topic that interests some people as they're still getting to know Python, and my most recent Stack Overflow question was from someone confused about some code using decorators and as recently as this morning I saw someone on #python say they "wanted to learn about decorators" in the abstract. In any event, I didn't mean _people completely brand new to Python_ when I talked about people who were new. > In my experience, most decorator factories (or factory-factories in general, > not just for generators but any time you need to wrap a factory function in > a closure) require at least one pre-processing step, and occasionally one or > two post-processing steps as well: In surveying actual decorator factories in the stdlib and my own code, I didn't notice it to be the case there was any preprocessing inside the outermost function. Maybe I just found the wrong ones. > Concise, yes, but I disagree that it is self-explaining. > > I'm having a lot of difficulty in thinking of how I would explain it to even > a moderately experienced user except by expanding it out to the explicit > nested function form. I'm not sure I can even explain it to myself. > > What will happen when the user invariably writes something like this? > > def ordinary(a)(b): > return a + b > > > My guess is that they will get a syntax error, but it will be a syntax error > in the wrong place: instead of clearly flagging the error (a)(b) as invalid > syntax, as you get now > > py> def ordinary(a)(b): > File "", line 1 > def ordinary(a)(b): > ^ > SyntaxError: invalid syntax > > > Instead, the user will get a less useful syntax error on the following line, > where the compiler sees that the function def is not followed immediately by > another function def. > > > py> def ordinary(a)(b): > ... return a + b > File "", line 1 > return a + b > ^ > SyntaxError: invalid syntax > > > So whatever clarity you (allegedly) gain when writing decorator factories, > you lose when making an error. I'm glad to get a different perspective here. It was evidently much more self-explanatory to the folks I discussed this with before posting. I wouldn't have expected this to require the def to be immediately followed by another def or for that to be the only (or most-common) case. I would expect that to be proper syntax under this idea, and for ordinary(3)(4) to be 7. I don't know if I said anything to suggest that the syntax should be quite so specialcased, but I hope not. Mike From jeanpierreda at gmail.com Mon Jul 9 03:45:11 2012 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Sun, 8 Jul 2012 21:45:11 -0400 Subject: [Python-ideas] Syntax for defining parametric decorators In-Reply-To: <4FFA2B3F.5010106@pearwood.info> References: <4FFA2B3F.5010106@pearwood.info> Message-ID: On Sun, Jul 8, 2012 at 8:52 PM, Steven D'Aprano wrote: > Concise, yes, but I disagree that it is self-explaining. I think the idea is that just as def foo(a): ... can be called as foo(x) to execute the body, def foo(a)(b): ... can be called as foo(x)(y) to execute the body. > I'm having a lot of difficulty in thinking of how I would explain it to even > a moderately experienced user except by expanding it out to the explicit > nested function form. I'm not sure I can even explain it to myself. What's wrong with explaining it by expansion? Also, I doubt it's as hard as you claim. Something similar -- curried functions -- have been a staple of functional programming languages since before Python ever existed. This should also help explain it to lots of new users with wider backgrounds. > What will happen when the user invariably writes something like this? > > def ordinary(a)(b): > return a + b Surely it'd define a curried function s.t. ordinary(1)(2) == 3? -- Devin From ncoghlan at gmail.com Mon Jul 9 03:46:16 2012 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 9 Jul 2012 11:46:16 +1000 Subject: [Python-ideas] Syntax for defining parametric decorators In-Reply-To: References: Message-ID: On Mon, Jul 9, 2012 at 11:11 AM, Mike Graham wrote: > On Sun, Jul 8, 2012 at 6:54 PM, Nick Coghlan wrote: >> Beyond that, no, this is too limited - it only helps when there's no extra >> code in the outer scope which, in my experience, is the exception rather >> than the rule. > > I'm not sure I think that's the case. - functools.wraps Not really, as it uses functools.partial, not a nested def (sure you could rewrite it to use a nested def instead, but why?) > functools.lru_cache No, as it has pre- and post-processing steps around the nested function definition. - reprlib.recursive_repr Yes, this one would qualify. However, the simplest possible complete definition of decorators is as follows: - a decorator expression (the bit after "@") on a def statement must produce a function decorator - a decorator expression on a class statement must produce a class decorator - a function decorator accepts a function and produces another object. This is typically another function (or even the original function for annotation and registration decorators), but may also be a different kind of object such as a descriptor (e.g. property, classmethod, staticmethod) or context manager (e.g. contextlib.contextmanager) - a class decorator is similar, but accepts a class rather than a function - a decorator factory is any callable that returns a decorator - function decorators are often written as functions that return a nested function (1 level of lexical nesting) - function decorator factories are often written as functions that return a function that returns a functions (2 levels of lexical nesting) Creating a dedicated syntax for the special case of function decorator factories written to use lexical nesting makes the language as a whole *more* complicated rather than less. The only way to actually make them *simpler* would be to eliminate one or more of the bullet points from the above list, and that can't be done while retaining the current functionality. Yes, it means that to write efficient custom decorator factories you need to understand both the decoration process and lexical closures. That's *OK* - it just means writing custom decorator factories (as opposed to using those written by others) is a moderately advanced metaprogramming technique (it's still a lot simpler than writing a custom metaclass, though). Some concepts are just hard to get your head around, but that doesn't mean we should be creating dedicated syntax for a very specialised use case. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From greg.ewing at canterbury.ac.nz Mon Jul 9 00:24:58 2012 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 09 Jul 2012 10:24:58 +1200 Subject: [Python-ideas] Syntax for defining parametric decorators In-Reply-To: References: Message-ID: <4FFA08BA.8090504@canterbury.ac.nz> Mike Graham wrote: > def timesn(n)(f): > def inner(y): > return n * f(y) > return inner +1 from me on this. Scheme has an analogous feature, btw, which is very elegant. I'd love to see this in Python. As has been pointed out, you can go further and collapse this into def timesn(n)(f)(y): return n * f(y) This means it would also be useful for non-parametric decorators as well, e.g. def fivetimes(f)(x): return 5 * f(x) -- Greg From jeanpierreda at gmail.com Mon Jul 9 03:55:09 2012 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Sun, 8 Jul 2012 21:55:09 -0400 Subject: [Python-ideas] Syntax for defining parametric decorators In-Reply-To: References: Message-ID: On Sun, Jul 8, 2012 at 9:46 PM, Nick Coghlan wrote: > However, the simplest possible complete definition of decorators is as follows: > > - a decorator expression (the bit after "@") on a def statement must > produce a function decorator > - a decorator expression on a class statement must produce a class decorator > - a function decorator accepts a function and produces another object. > This is typically another function (or even the original function for > annotation and registration decorators), but may also be a different > kind of object such as a descriptor (e.g. property, classmethod, > staticmethod) or context manager (e.g. contextlib.contextmanager) > - a class decorator is similar, but accepts a class rather than a function > - a decorator factory is any callable that returns a decorator > - function decorators are often written as functions that return a > nested function (1 level of lexical nesting) > - function decorator factories are often written as functions that > return a function that returns a functions (2 levels of lexical > nesting) Mmm, this distinction of "kinds of decorators" is silly. The only requirement for something to be a decorator is that it takes one argument. The type of the argument could be anything. def dumb_decorator(x): return x + 2 @dumb_decorator @apply def foo(): return 5 > Creating a dedicated syntax for the special case of function decorator > factories written to use lexical nesting makes the language as a whole > *more* complicated rather than less. The only way to actually make > them *simpler* would be to eliminate one or more of the bullet points > from the above list, and that can't be done while retaining the > current functionality. Nothing about the OP's suggestion was specific to "function decorator factories". Observe a class decorator factory: def stupid_augmented_subclass_decorator(**kwargs)(cls): class MyClass(cls): pass MyClass.__dict__.update(**kwargs) return MyClass -- Devin From ncoghlan at gmail.com Mon Jul 9 04:00:41 2012 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 9 Jul 2012 12:00:41 +1000 Subject: [Python-ideas] Syntax for defining parametric decorators In-Reply-To: References: <4FFA2B3F.5010106@pearwood.info> Message-ID: On Mon, Jul 9, 2012 at 11:45 AM, Devin Jeanpierre wrote: >> I'm having a lot of difficulty in thinking of how I would explain it to even >> a moderately experienced user except by expanding it out to the explicit >> nested function form. I'm not sure I can even explain it to myself. > > What's wrong with explaining it by expansion? > > Also, I doubt it's as hard as you claim. Something similar -- curried > functions -- have been a staple of functional programming languages > since before Python ever existed. This should also help explain it to > lots of new users with wider backgrounds. OK, I think it would be *much* better to approach the problem from that angle. def incremental(x)(f)(y): return x + f(y) As equivalent to: # implied names are not legal identifiers - the compiler gets to do that # because of its privileged role in naming things def incremental(x): def (f): def (y): return x +f(y) return I'm still not convinced of the general applicability (since it breaks down as soon as you want to decorate or otherwise pre- or post-process any of the steps, thus forcing people to learn the full "callable-returning-a-callable" idiom anyway), but it's probably worth writing up as a PEP in the 3.4 timeframe. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Mon Jul 9 04:02:48 2012 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 9 Jul 2012 12:02:48 +1000 Subject: [Python-ideas] Syntax for defining parametric decorators In-Reply-To: References: Message-ID: On Mon, Jul 9, 2012 at 11:55 AM, Devin Jeanpierre wrote: > Nothing about the OP's suggestion was specific to "function decorator > factories". The sole motivating use case presented was "A common stumbling block for new users is writing decorators that take arguments." It's a terrible motivating use case for curried functions, because they only work for the most trivial of decorator factories. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From haoyi.sg at gmail.com Mon Jul 9 04:04:14 2012 From: haoyi.sg at gmail.com (Haoyi Li) Date: Sun, 8 Jul 2012 19:04:14 -0700 Subject: [Python-ideas] Syntax for defining parametric decorators In-Reply-To: References: Message-ID: Just another point of data, Scala has exactly what is being proposed: multiple parameter lists as synctactic sugar for nested return functions http://stackoverflow.com/questions/4697404/scala-currying-by-nested-functions-or-by-multiple-parameter-lists It also works with all the non-trivial edge-casey stuff python has (args, default/named args, varargs, kwargs, splats). My experience with this is that it actually works pretty well, the number of "function returning function" types which require pre- or post- processing is relatively small, and often this pre/post processing can simply be moved into the body of the function for no real cost. The cases where that cannot be done, it usually is because apart from simply returning a new function, I'm also enclosing over some mutable state in the outer function. In these cases I've found that this implicitly captured mutable state is better handled with an object-based decorator, making the stateful nature explicit. The lru_cache decorator seems like a good candidate for using explicit objects to hold the cache, rather cunningly enclosed mutable state. I agree that as a "decorator factory helper", having extra syntax is unwarranted. However, I think that in the general case, being able to define curried functions in one line, a.l.a. def func(a)(b)(c) = a * b * c rather than def func(a): def wrapper_two(b): def wrapper_one(c): return a * b * c return wrapper_one return wrapper_two would in fact be very nice. From the Zen of Python, I would say these are big wins for - Flat is better than nested - Readability Counts - Beautiful is better than ugly whether it's a win or lose for Explicit is better than Implicit is debatable: In the implementation domain, you're being less explicit about what's actually happening (function returning functions) but in the "how-to-use-me" domain you're being more explicit about what you actually want (call this function with multiple parameter lists). def f(a)(b)(c) looks much more similar to how you would use it: f(1)(2)(3) than that horrible triple-nested monstrosity above. -Haoyi On Sun, Jul 8, 2012 at 6:46 PM, Nick Coghlan wrote: > On Mon, Jul 9, 2012 at 11:11 AM, Mike Graham wrote: > > On Sun, Jul 8, 2012 at 6:54 PM, Nick Coghlan wrote: > >> Beyond that, no, this is too limited - it only helps when there's no > extra > >> code in the outer scope which, in my experience, is the exception rather > >> than the rule. > > > > I'm not sure I think that's the case. > > - functools.wraps > > Not really, as it uses functools.partial, not a nested def (sure you > could rewrite it to use a nested def instead, but why?) > > > functools.lru_cache > > No, as it has pre- and post-processing steps around the nested > function definition. > > - reprlib.recursive_repr > > Yes, this one would qualify. > > However, the simplest possible complete definition of decorators is as > follows: > > - a decorator expression (the bit after "@") on a def statement must > produce a function decorator > - a decorator expression on a class statement must produce a class > decorator > - a function decorator accepts a function and produces another object. > This is typically another function (or even the original function for > annotation and registration decorators), but may also be a different > kind of object such as a descriptor (e.g. property, classmethod, > staticmethod) or context manager (e.g. contextlib.contextmanager) > - a class decorator is similar, but accepts a class rather than a function > - a decorator factory is any callable that returns a decorator > - function decorators are often written as functions that return a > nested function (1 level of lexical nesting) > - function decorator factories are often written as functions that > return a function that returns a functions (2 levels of lexical > nesting) > > Creating a dedicated syntax for the special case of function decorator > factories written to use lexical nesting makes the language as a whole > *more* complicated rather than less. The only way to actually make > them *simpler* would be to eliminate one or more of the bullet points > from the above list, and that can't be done while retaining the > current functionality. > > Yes, it means that to write efficient custom decorator factories you > need to understand both the decoration process and lexical closures. > That's *OK* - it just means writing custom decorator factories (as > opposed to using those written by others) is a moderately advanced > metaprogramming technique (it's still a lot simpler than writing a > custom metaclass, though). > > Some concepts are just hard to get your head around, but that doesn't > mean we should be creating dedicated syntax for a very specialised use > case. > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wuwei23 at gmail.com Mon Jul 9 06:16:51 2012 From: wuwei23 at gmail.com (alex23) Date: Sun, 8 Jul 2012 21:16:51 -0700 (PDT) Subject: [Python-ideas] itertools recipes: why not add them to the stdlib *somewhere*? In-Reply-To: <20120706181404.GB3347@jaerhard.com> References: <20120706181404.GB3347@jaerhard.com> Message-ID: <206b7fef-3f0a-48c5-b994-a0e75cfcac0e@vb9g2000pbc.googlegroups.com> On Jul 7, 4:14?am, J?rgen A. Erhard wrote: > So... I consider it very strange and annoying that there's this code. > Ready to use. ?But you have to copy-and-paste it into your code > *somewhere* instead of easily importing it. I find it strange and annoying when people expect the standard library to contain _everything_. What you gain in the convenience of readily imported code imposes a cost on the ongoing maintenance and support of the standard lib. Are you volunteering to write the PEP for its inclusion, provide the implementation and support it until the end of time? > As to the "it makes using Python harder to learn", I beg to > differ: an addition to the *language* (like, say, metaclasses) *can* > make it more complicated. ?But additions to the stdlib? ?What about > "batteries included"? ?Not a motto anymore (I heard rumors, so maybe > that's the case) Every single function added to the library makes the library broader and thus more difficult to retain in memory. It _is_ possible to present a case without snide allusions based on hearsay: http://www.python.org/about/ > Putting these in some official(!) iterutils (or name it what you want) > package would be a solution for so many people. ?Yes, not everyone > needs grouper's current functionality. ?But for the many who do, it'd > be there already. So in the extremely likely outcome that a piece of functionality has multiple implementations with different interpretations of edge cases, who gets to decide which one is "more standard" than the other? If such a decision has to be made, then it's not really "standard". > And someone thought those recipes are useful, > didn't you? ?So why not make them more easily available? Because not everything needs to be in the standard library. Because searching somewhere like Activestate's cookbook and copying the implementations you find useful really isn't an onerous task. Because if *you* feel there's an obvious lack in the stdlib then roll your own package that covers it and put it up on PyPI. If it becomes insanely popular, then you have a lot more leverage for insisting that something would be beneficial to be included other than gut feeling & convenience. From ncoghlan at gmail.com Mon Jul 9 06:58:06 2012 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 9 Jul 2012 14:58:06 +1000 Subject: [Python-ideas] itertools recipes: why not add them to the stdlib *somewhere*? In-Reply-To: <206b7fef-3f0a-48c5-b994-a0e75cfcac0e@vb9g2000pbc.googlegroups.com> References: <20120706181404.GB3347@jaerhard.com> <206b7fef-3f0a-48c5-b994-a0e75cfcac0e@vb9g2000pbc.googlegroups.com> Message-ID: On Mon, Jul 9, 2012 at 2:16 PM, alex23 wrote: > So in the extremely likely outcome that a piece of functionality has > multiple implementations with different interpretations of edge cases, > who gets to decide which one is "more standard" than the other? If > such a decision has to be made, then it's not really "standard". This is often what lies at the heart of "not every 3 line function needs to be /". It's very easy to hit a point of diminishing returns where the number of possible alternatives mean that attempting to create an abstraction layer ends up creating a UI that is *more complicated* than just writing your own utility function that does exactly what you want. Grouping is one such API (fixed length? Variable length with a delimiter? Pad last group? Drop last group? Error early? Accept arbitrary iterables? Accept sequences only? Support overlapping windows?). Recursive descent into arbitrary collections is another (Error on reference loops? Treat strings/bytes/bytearray as atomic types? Treat any implementor of the buffer interface as an atomic type? Support only hashable objects?). There are a few key reasons why things end up in the standard library: 1. They're closely coupled to the language definition and implementation and should be updated in sync with it (e.g. sys, imp, gc, importlib, dis, inspect) 2. We want to use them in other parts of the standard library or its test suite (e.g. collections, many of the unittest enhancements) 3. They solve a common problem that is otherwise prone to being handled with incomplete solutions that lead to incorrect code (e.g. ipaddress instead of regex based recipes for processing IP addresses) 4. It's an old module that would probably be left to PyPI these days, but was added in earlier times when stdlib inclusion criteria were less strict (but isn't broken so there's no harm in keeping it around). Sometimes a problem is sufficiently common, and has few enough variations, that it's worth creating and providing a standard version. That's the general aim of the itertools module. Other times, the core problem is difficult enough that it's worth providing a standard solution, even if it does mean including a vast array of configuration options (e.g. subprocess. Popen). However, sometimes, the correct answer to "Hey, this is a really common pattern" is not "We should provide an API that uses that pattern internally" but "we should document this pattern, so people know it's a common idiom and can tailor it to their specific use case and preferences". Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From stephen at xemacs.org Mon Jul 9 07:48:37 2012 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 09 Jul 2012 14:48:37 +0900 Subject: [Python-ideas] Syntax for defining parametric decorators In-Reply-To: References: Message-ID: <87obnp5vt6.fsf@uwakimon.sk.tsukuba.ac.jp> Haoyi Li writes: > However, I think that in the general case, being able to define > curried functions in one line, a.l.a. [...] > def f(a)(b)(c) > > looks much more similar to how you would use it: > > f(1)(2)(3) I don't understand. I wouldn't use it that way! I would think if that were common usage, and the curried form only occasional, def f(a,b,c): pass f(1,2,3) would be more natural in Python (and use functools.partial for the occasional currying). Presumably if you were using this syntax, you'd be doing something more like def f(a)(b)(c): pass def g(h, x): h(x) g(f(1)(2), 3) which doesn't look very nice to me, nor do I find the def to be a particularly intuitive way of reminding me that the proper usage is a call of a function in curried form (specifically, I would not be reminded of whether the expected curried form is a 1st-, 2nd-, or 3rd-order function). I also suppose that you wouldn't be able to do def f(a)(b)(c): pass def g(x,y): pass h(f(1)) h(g) (ie, without LBYL or EAFP constructs in h, which would simply be throwing the complexity into the caller's backyard). Even though def bar(a)(b): pass def baz(x): pass quux(bar(1)) quux(baz) would work fine. So to me, there may be something in this syntax, but my initial impression is that it's trying to turn Python into something it's not. From haoyi.sg at gmail.com Mon Jul 9 11:07:49 2012 From: haoyi.sg at gmail.com (Haoyi Li) Date: Mon, 9 Jul 2012 02:07:49 -0700 Subject: [Python-ideas] Syntax for defining parametric decorators Message-ID: <2237588560696344575@unknownmsgid> I admit calling it all at once would be very strange, but i still think the "call me three times to get result" comes across waaaay more clearly than with nested defs. Multiple arguments in one list is about the same, it's true, until you really need the curried version, like in decorators, then you end up with the nasty tower-of-defs which sparked this discussion. Regarding "not knowing how many times it should be called", the idea is that you break it up when you have a meaningful, reusable unit. That's to say you wouldn't do a def f(a)(b)(c) Unless f f(1) f(1)(2) f(1)(2)(3) Are all meaningful, useful functions. Doesn't happen very often, granted, but when it does (e.g. for decorator factories and decorators) it'd be nice to have. I view curried defs like default parameters: Doesnt add anything that cant already be done (you can emulate default args with telescoping methods) but cuts boilerplate, makes things clearer and overall nice to have. Whether it's nice enough to justify extra syntax is of course open for debate. From: Stephen J. Turnbull Sent: 7/9/2012 1:48 AM To: Haoyi Li Cc: Nick Coghlan; Python-Ideas Subject: Re: [Python-ideas] Syntax for defining parametric decorators Haoyi Li writes: > However, I think that in the general case, being able to define > curried functions in one line, a.l.a. [...] > def f(a)(b)(c) > > looks much more similar to how you would use it: > > f(1)(2)(3) I don't understand. I wouldn't use it that way! I would think if that were common usage, and the curried form only occasional, def f(a,b,c): pass f(1,2,3) would be more natural in Python (and use functools.partial for the occasional currying). Presumably if you were using this syntax, you'd be doing something more like def f(a)(b)(c): pass def g(h, x): h(x) g(f(1)(2), 3) which doesn't look very nice to me, nor do I find the def to be a particularly intuitive way of reminding me that the proper usage is a call of a function in curried form (specifically, I would not be reminded of whether the expected curried form is a 1st-, 2nd-, or 3rd-order function). I also suppose that you wouldn't be able to do def f(a)(b)(c): pass def g(x,y): pass h(f(1)) h(g) (ie, without LBYL or EAFP constructs in h, which would simply be throwing the complexity into the caller's backyard). Even though def bar(a)(b): pass def baz(x): pass quux(bar(1)) quux(baz) would work fine. So to me, there may be something in this syntax, but my initial impression is that it's trying to turn Python into something it's not. From stefan_ml at behnel.de Mon Jul 9 11:47:48 2012 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 09 Jul 2012 11:47:48 +0200 Subject: [Python-ideas] Syntax for defining parametric decorators In-Reply-To: References: Message-ID: Mike Graham, 08.07.2012 22:22: > A common stumbling block for new users is writing decorators that take > arguments. To create a decorator like > > @timesn(n) > def f(y): > ... > > We write code like > > def timesn(n) > def decorator(f): > def inner(y): > return n * f(y) > return inner > return decorator > > which confuses many users and can be a handful to type. I wonder if it > would be clearer for people to write > > def timesn(n)(f): > def inner(y): > return n * f(y) > return inner > > which is more concise and looks a lot more like a non-parametric > decorator someone might have written already. The syntax is mostly > self-explaining and could potentially be useful in other contexts. >From an innocent look, I have no idea what the syntax is supposed to mean. Clearly doesn't hint at a factory for me. Stefan From mwm at mired.org Mon Jul 9 11:56:55 2012 From: mwm at mired.org (Mike Meyer) Date: Mon, 9 Jul 2012 05:56:55 -0400 Subject: [Python-ideas] Syntax for defining parametric decorators In-Reply-To: References: Message-ID: <20120709055655.49a978c7@bhuda.mired.org> On Mon, 09 Jul 2012 11:47:48 +0200 Stefan Behnel wrote: > Mike Graham, 08.07.2012 22:22: > > def timesn(n)(f): > > def inner(y): > > return n * f(y) > > return inner > > > > which is more concise and looks a lot more like a non-parametric > > decorator someone might have written already. The syntax is mostly > > self-explaining and could potentially be useful in other contexts. > > >From an innocent look, I have no idea what the syntax is supposed to mean. > Clearly doesn't hint at a factory for me. Does to me. timesn is clearly a function of one argument that is going to return a callable object that takes one argument. So: x5 = times(5) print x5(7) should print 35. I'd be interested in seeing an implementation of partial that used this mechanism. Should be straightforward, but I'm in the midst of a crunch. http://www.mired.org/ Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From steve at pearwood.info Mon Jul 9 13:32:59 2012 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 09 Jul 2012 21:32:59 +1000 Subject: [Python-ideas] itertools recipes: why not add them to the stdlib *somewhere*? In-Reply-To: References: <20120706181404.GB3347@jaerhard.com> <206b7fef-3f0a-48c5-b994-a0e75cfcac0e@vb9g2000pbc.googlegroups.com> Message-ID: <4FFAC16B.7030206@pearwood.info> Nick Coghlan wrote: > It's very easy to hit a point of diminishing returns where the number > of possible alternatives mean that attempting to create an abstraction > layer ends up creating a UI that is *more complicated* than just > writing your own utility function that does exactly what you want. [...] +1 on everything Nick says here. One additional comment: > However, sometimes, the correct answer to "Hey, this is a really > common pattern" is not "We should provide an API that uses that > pattern internally" but "we should document this pattern, so people > know it's a common idiom and can tailor it to their specific use case > and preferences". Well, maybe, but documenting software patterns are not necessarily "our" job (whoever "our" is -- the Python devs, or python.org, or the PSI, or whoever people think is responsible). Python has not been a one-man project for many, many years. There is a rich Python ecosystem now, not just Guido or the Python devs, but StackOverflow, comp.lang.python, dozens of books, more bloggers than you can poke a stick at, #python, etc etc etc. It's not "our" responsibility to teach every Python programmer every conceivable pattern. At some point we can just draw a line and say "Yeah, that's a useful pattern, but we're not going to document it. Somebody else can do it." Even if that line is somewhat arbitrary. -- Steven From ironfroggy at gmail.com Mon Jul 9 14:28:58 2012 From: ironfroggy at gmail.com (Calvin Spealman) Date: Mon, 9 Jul 2012 08:28:58 -0400 Subject: [Python-ideas] Syntax for defining parametric decorators In-Reply-To: References: Message-ID: On Sun, Jul 8, 2012 at 4:22 PM, Mike Graham wrote: > A common stumbling block for new users is writing decorators that take > arguments. To create a decorator like > > @timesn(n) > def f(y): > ... > > We write code like > > def timesn(n) > def decorator(f): > def inner(y): > return n * f(y) > return inner > return decorator > > which confuses many users and can be a handful to type. I wonder if it > would be clearer for people to write > > def timesn(n)(f): > def inner(y): > return n * f(y) > return inner > > which is more concise and looks a lot more like a non-parametric > decorator someone might have written already. The syntax is mostly > self-explaining and could potentially be useful in other contexts. > > There exist tools like the decorator library to try to simplify this > already, but in my experience they mostly serve to confuse people > using decorators for the first time more. > > One thing I didn't specify was whether `n` was nonlocal or not and the > behavior of something that keeps and reuses timesn(some_specific_n) > multiple times. > > Does anyone think a feature like this may be useful? While I'm a bit taken aback by the syntax, I recognize this is mostly because I'm not used to it. Objectively, it makes enough sense that I think it fits. I think even if the only use case is decorator factories, they are a useful enough thing that this is okay. We have a specific syntax to support using decorators, why not better support writing them? Still, I'd like to see any ideas people have for alternative uses for this syntax. > Regards, > Mike > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy From jsbueno at python.org.br Mon Jul 9 15:14:26 2012 From: jsbueno at python.org.br (Joao S. O. Bueno) Date: Mon, 9 Jul 2012 10:14:26 -0300 Subject: [Python-ideas] Syntax for defining parametric decorators In-Reply-To: References: Message-ID: On 9 July 2012 09:28, Calvin Spealman wrote: > On Sun, Jul 8, 2012 at 4:22 PM, Mike Graham wrote: >> A common stumbling block for new users is writing decorators that take >> arguments. To create a decorator like >> >> @timesn(n) >> def f(y): >> ... -1 for the new syntax, since a decorator for trivial decorator factories (rather, parameterized decorators) can be written in a total of 5 lines: from functools import partial def parameterized(decorator): def part_decorator(*args, **kw): return partial(decorator, *args, **kw) return part_decorator >>> @parameterized ... def times(n, func): ... def new_func(*args, **kw): ... return n * func(*args, **kw) ... return new_func ... >>> >>> @times(3) ... def add(x ,y): ... return x + y ... >>> add(1,1) 6 ------- For more complex cases, requiring pre processing, post processing, and so on, the normal syntax can cut it. And, certainly, a more complex such decorator could be written, so that it accepts some arguments itself. (I would be +1 for such a decorator in functools) js -><- From aquavitae69 at gmail.com Mon Jul 9 17:18:23 2012 From: aquavitae69 at gmail.com (David Townshend) Date: Mon, 9 Jul 2012 17:18:23 +0200 Subject: [Python-ideas] itertools recipes: why not add them to the stdlib *somewhere*? In-Reply-To: <206b7fef-3f0a-48c5-b994-a0e75cfcac0e@vb9g2000pbc.googlegroups.com> References: <20120706181404.GB3347@jaerhard.com> <206b7fef-3f0a-48c5-b994-a0e75cfcac0e@vb9g2000pbc.googlegroups.com> Message-ID: On Jul 9, 2012 6:17 AM, "alex23" wrote: > > On Jul 7, 4:14 am, J?rgen A. Erhard wrote: > > So... I consider it very strange and annoying that there's this code. > > Ready to use. But you have to copy-and-paste it into your code > > *somewhere* instead of easily importing it. > > I find it strange and annoying when people expect the standard library > to contain _everything_. What you gain in the convenience of readily > imported code imposes a cost on the ongoing maintenance and support of > the standard lib. Are you volunteering to write the PEP for its > inclusion, provide the implementation and support it until the end of > time? While I agree that these recipes are not suitable for inclusion in stdlib (for reasons made elsewhere), I don't agree with your (somewhat aggressive) reasoning. The implementation has already been provided, and is being maintained in the form of documentation. Simply copying the recipes into a module is hardly an onerous task. And fairly often people mailing to this list *are* volunteering to write a PEP, provide an implementation, etc. > > > As to the "it makes using Python harder to learn", I beg to > > differ: an addition to the *language* (like, say, metaclasses) *can* > > make it more complicated. But additions to the stdlib? What about > > "batteries included"? Not a motto anymore (I heard rumors, so maybe > > that's the case) > > Every single function added to the library makes the library broader > and thus more difficult to retain in memory. > > It _is_ possible to present a case without snide allusions based on > hearsay: http://www.python.org/about/ > > > Putting these in some official(!) iterutils (or name it what you want) > > package would be a solution for so many people. Yes, not everyone > > needs grouper's current functionality. But for the many who do, it'd > > be there already. > > So in the extremely likely outcome that a piece of functionality has > multiple implementations with different interpretations of edge cases, > who gets to decide which one is "more standard" than the other? If > such a decision has to be made, then it's not really "standard". > > > And someone thought those recipes are useful, > > didn't you? So why not make them more easily available? > > Because not everything needs to be in the standard library. > > Because searching somewhere like Activestate's cookbook and copying > the implementations you find useful really isn't an onerous task. > > Because if *you* feel there's an obvious lack in the stdlib then roll > your own package that covers it and put it up on PyPI. If it becomes > insanely popular, then you have a lot more leverage for insisting that > something would be beneficial to be included other than gut feeling & > convenience. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas -------------- next part -------------- An HTML attachment was scrubbed... URL: From raymond.hettinger at gmail.com Mon Jul 9 18:18:54 2012 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Mon, 9 Jul 2012 09:18:54 -0700 Subject: [Python-ideas] itertools recipes: why not add them to the stdlib *somewhere*? In-Reply-To: References: <20120706181404.GB3347@jaerhard.com> <206b7fef-3f0a-48c5-b994-a0e75cfcac0e@vb9g2000pbc.googlegroups.com> Message-ID: On Jul 9, 2012, at 8:18 AM, David Townshend wrote: > . Simply copying the recipes into a module is hardly an onerous task. In fact, it has already been done :-) http://pypi.python.org/pypi?%3Aaction=search&term=itertools&submit=search Raymond -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Mon Jul 9 21:47:10 2012 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 09 Jul 2012 15:47:10 -0400 Subject: [Python-ideas] itertools recipes: why not add them to the stdlib *somewhere*? In-Reply-To: References: <20120706181404.GB3347@jaerhard.com> <206b7fef-3f0a-48c5-b994-a0e75cfcac0e@vb9g2000pbc.googlegroups.com> Message-ID: On 7/9/2012 12:18 PM, Raymond Hettinger wrote: > > On Jul 9, 2012, at 8:18 AM, David Townshend wrote: > >> . Simply copying the recipes into a module is hardly an onerous task. > > In fact, it has already been done :-) Multiple times. Possible addition to the doc in 9.1.2 after first sentence: These and similar recipes have been incorporated in various third-party modules. See ... > http://pypi.python.org/pypi?%3Aaction=search&term=itertools&submit=search -- Terry Jan Reedy From jeanpierreda at gmail.com Mon Jul 9 22:12:22 2012 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Mon, 9 Jul 2012 16:12:22 -0400 Subject: [Python-ideas] itertools recipes: why not add them to the stdlib *somewhere*? In-Reply-To: <206b7fef-3f0a-48c5-b994-a0e75cfcac0e@vb9g2000pbc.googlegroups.com> References: <20120706181404.GB3347@jaerhard.com> <206b7fef-3f0a-48c5-b994-a0e75cfcac0e@vb9g2000pbc.googlegroups.com> Message-ID: On Mon, Jul 9, 2012 at 12:16 AM, alex23 wrote: > I find it strange and annoying when people expect the standard library > to contain _everything_. What you gain in the convenience of readily > imported code imposes a cost on the ongoing maintenance and support of > the standard lib. Are you volunteering to write the PEP for its > inclusion, provide the implementation and support it until the end of > time? Are you saying the recipes are unsupported? -- Devin From wuwei23 at gmail.com Tue Jul 10 00:28:41 2012 From: wuwei23 at gmail.com (wu wei) Date: Tue, 10 Jul 2012 08:28:41 +1000 Subject: [Python-ideas] itertools recipes: why not add them to the stdlib *somewhere*? In-Reply-To: References: <20120706181404.GB3347@jaerhard.com> <206b7fef-3f0a-48c5-b994-a0e75cfcac0e@vb9g2000pbc.googlegroups.com> Message-ID: On Tue, Jul 10, 2012 at 6:12 AM, Devin Jeanpierre wrote: > Are you saying the recipes are unsupported? > This is getting ridiculous. I'm not sure what is so controversial about not wanting to scrape up every piece of crap code off the web that some lazy coder can't be bothered to implement contextually and shove them into the _standard library_. Are _you_ seriously saying that every one of those random bits of code _is_ supported? That they will all be kept up to date as the language evolves? That they won't turn into another module that people complain is "stale"? Or another "why tkinter and not " argument? Every inclusion to the library adds overhead to finding and using elements of the library. -------------- next part -------------- An HTML attachment was scrubbed... URL: From phd at phdru.name Tue Jul 10 00:36:26 2012 From: phd at phdru.name (Oleg Broytman) Date: Tue, 10 Jul 2012 02:36:26 +0400 Subject: [Python-ideas] itertools recipes: why not add them to the stdlib *somewhere*? In-Reply-To: References: <20120706181404.GB3347@jaerhard.com> <206b7fef-3f0a-48c5-b994-a0e75cfcac0e@vb9g2000pbc.googlegroups.com> Message-ID: <20120709223626.GB1279@iskra.aviel.ru> On Tue, Jul 10, 2012 at 08:28:41AM +1000, wu wei wrote: > Every inclusion to the library adds overhead to finding and using elements > of the library. And supporting it. It's much harder to remove bits from stdlib than to add to it because stdlib have to maintain backward compatibility. Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From alexandre.zani at gmail.com Tue Jul 10 00:42:52 2012 From: alexandre.zani at gmail.com (Alexandre Zani) Date: Mon, 9 Jul 2012 15:42:52 -0700 Subject: [Python-ideas] itertools recipes: why not add them to the stdlib *somewhere*? In-Reply-To: <20120709223626.GB1279@iskra.aviel.ru> References: <20120706181404.GB3347@jaerhard.com> <206b7fef-3f0a-48c5-b994-a0e75cfcac0e@vb9g2000pbc.googlegroups.com> <20120709223626.GB1279@iskra.aviel.ru> Message-ID: On Mon, Jul 9, 2012 at 3:36 PM, Oleg Broytman wrote: > On Tue, Jul 10, 2012 at 08:28:41AM +1000, wu wei wrote: >> Every inclusion to the library adds overhead to finding and using elements >> of the library. > > And supporting it. It's much harder to remove bits from stdlib than > to add to it because stdlib have to maintain backward compatibility. > > Oleg. > -- > Oleg Broytman http://phdru.name/ phd at phdru.name > Programmers don't die, they just GOSUB without RETURN. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas It is a good point that those recipes because they are in the official docs are already requiring support and maintenance. However, as somebody above mentioned, there are many possible variations on those recipes and so it makes a lot more sense to tell people to implement their own version of grouper that will match their specific needs rather than create an infinitely configurable version of grouper in the stdlib. Furthermore, if the maintenance of those recipes ever becomes undesirable, it is a simple matter to strike them from the docs. The same cannot be said if they were inducted in the stdlib. Alexandre Zani From jeanpierreda at gmail.com Tue Jul 10 00:45:55 2012 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Mon, 9 Jul 2012 18:45:55 -0400 Subject: [Python-ideas] itertools recipes: why not add them to the stdlib *somewhere*? In-Reply-To: References: <20120706181404.GB3347@jaerhard.com> <206b7fef-3f0a-48c5-b994-a0e75cfcac0e@vb9g2000pbc.googlegroups.com> Message-ID: On Mon, Jul 9, 2012 at 6:28 PM, wu wei wrote: > On Tue, Jul 10, 2012 at 6:12 AM, Devin Jeanpierre > wrote: >> >> Are you saying the recipes are unsupported? > > > This is getting ridiculous. I'm not sure what is so controversial about not > wanting to scrape up every piece of crap code off the web that some lazy This is code from the official documentation. > coder can't be bothered to implement contextually and shove them into the > _standard library_. Are _you_ seriously saying that every one of those > random bits of code _is_ supported? No, I am suggesting that example code that exists in the documentation is supported. I think you are confused. -- Devin From tjreedy at udel.edu Tue Jul 10 03:52:55 2012 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 09 Jul 2012 21:52:55 -0400 Subject: [Python-ideas] itertools recipes: why not add them to the stdlib *somewhere*? In-Reply-To: References: <20120706181404.GB3347@jaerhard.com> <206b7fef-3f0a-48c5-b994-a0e75cfcac0e@vb9g2000pbc.googlegroups.com> Message-ID: On 7/9/2012 6:45 PM, Devin Jeanpierre wrote: > No, I am suggesting that example code that exists in the documentation > is supported. Well, in the sense that we try to make sure that they work as advertised *with the example data*, yes. However, that is the end of the support. Examples are just examples, put there to illustrate a didactic point. They are often simplified to serve the specific purpose. They are not intended to be production code, and they are not supported as that. Error handling is eliminated for clarity.# The api has not necessarily been thought through very well and is subject to change.# The multiple examples are not particularly meant to work together. #Whereas, for example, the new in 3.3 ipaddress module, while based on existing 3rd party code, has had its api reexamined to make sure it well defined, documented, and something we will (likely) stick with. Nick has added a lot of error checking code with messages that try to say what is wrong rather than just 'you goofed'. And oh yes, the test suite has been expanded and reviewed with an aim to 100% coverage. -- Terry Jan Reedy From jeanpierreda at gmail.com Tue Jul 10 05:38:38 2012 From: jeanpierreda at gmail.com (Devin Jeanpierre) Date: Mon, 9 Jul 2012 23:38:38 -0400 Subject: [Python-ideas] itertools recipes: why not add them to the stdlib *somewhere*? In-Reply-To: References: <20120706181404.GB3347@jaerhard.com> <206b7fef-3f0a-48c5-b994-a0e75cfcac0e@vb9g2000pbc.googlegroups.com> Message-ID: On Mon, Jul 9, 2012 at 9:52 PM, Terry Reedy wrote: > Well, in the sense that we try to make sure that they work as advertised > *with the example data*, yes. However, that is the end of the support. > Examples are just examples, put there to illustrate a didactic point. They > are often simplified to serve the specific purpose. They are not intended to > be production code, and they are not supported as that. I've never assumed that (obviously, I guess), and that isn't stated in the docs. If that's the level of support of those recipes, this should be made clear inside the itertools documentation, because they _are_ used in peoples' code. For example, see the third party module linked earlier in the thread. On the contrary, the docs state the recipes are for creating an "extended toolset", implying that they can be used freely the same as the rest of the code. -- Devin From stephen at xemacs.org Tue Jul 10 06:39:50 2012 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 10 Jul 2012 13:39:50 +0900 Subject: [Python-ideas] itertools recipes: why not add them to the stdlib *somewhere*? In-Reply-To: References: <20120706181404.GB3347@jaerhard.com> <206b7fef-3f0a-48c5-b994-a0e75cfcac0e@vb9g2000pbc.googlegroups.com> Message-ID: <87fw905iw9.fsf@uwakimon.sk.tsukuba.ac.jp> Devin Jeanpierre writes: > I've never assumed that (obviously, I guess), and that isn't stated in > the docs. If that's the level of support of those recipes, this should > be made clear inside the itertools documentation, It already is clear. The very name "recipe" indicates that it may not be to everyone's taste, that it's not carefully designed to handle corner cases, and that people should review the code for their own use. The (very sparse) documentation associated with the recipes, which is clearly oriented to advising you how you can (a) produce similar code for different functions yourself and (b) make it better further emphasizes what the name indicates. Of course, all that may not be obvious to non-native speakers, but that's a completely different issue from the starting point of this thread, which was that these recipes belong in the stdlib rather than in its documentation. I have nothing to add to Terry's post explaining why that is not appropriate for these recipes "as is." > because they _are_ used in peoples' code. Of course they are. That's what they're there for. From techtonik at gmail.com Tue Jul 10 08:22:11 2012 From: techtonik at gmail.com (anatoly techtonik) Date: Tue, 10 Jul 2012 09:22:11 +0300 Subject: [Python-ideas] Explanation on how to search the archives In-Reply-To: <4FF887A7.5070101@pearwood.info> References: <4FF887A7.5070101@pearwood.info> Message-ID: On Sat, Jul 7, 2012 at 10:01 PM, Steven D'Aprano wrote: > > Teaching people how to search the Internet and > interact with mailing lists without being annoying is not the job of the > Python devs Aren't they primarily Python devs who are annoyed and complain that people don't search properly? I don't think your proposal to just write somewhere on the Internet, but not on python.org helps. Even wiki.python.org doesn't help if people don't have a chance to read it before posting. The idea about giving a convenient search tool is pretty valid. The true problem is that *.python.org is unable to do anything (not even a search form here - http://mail.python.org/mailman/listinfo/python-ideas). For usenet there were auto announcements, for Google Groups and forums there are sticky threads with advices. -- anatoly t. From ncoghlan at gmail.com Tue Jul 10 08:32:24 2012 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 10 Jul 2012 16:32:24 +1000 Subject: [Python-ideas] Syntax for defining parametric decorators In-Reply-To: References: Message-ID: On Mon, Jul 9, 2012 at 7:47 PM, Stefan Behnel wrote: > >From an innocent look, I have no idea what the syntax is supposed to mean. > Clearly doesn't hint at a factory for me. I should also mention that I have a different proposal that affects the way one would write functions-that-returns-functions. I've been messing around with the idea of statement local namespaces for years (see PEP 3150) trying to find something that I consider better than the status quo, and PEP 403's statement local function and class definitions (http://www.python.org/dev/peps/pep-0403/) are the current incarnation. With those, the number of statements in a simple wrapping decorator factory doesn't change, but the return statements can be moved above their respective function definitions: def notify_on_call(callback, *cb_args, **cb_kwds): in return decorator def decorator(f): in return wrapped @functools.wraps(f) def wrapped(*args, **kwds): callback(*cb_args, cb_kwds) return f(*args, **kwds) Rather than the current out-of-order: def notify_on_call(callback, *cb_args, **cb_kwds): def decorator(f): @functools.wraps(f) def wrapped(*args, **kwds): callback(*cb_args, cb_kwds) return f(*args, **kwds) return wrapped return decorator (Note: I haven't updated the PEP in a while, so it currently still disallows combining the in statement with decorators - I think that's a mistake, and will be updating it some time post 3.3) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From techtonik at gmail.com Tue Jul 10 09:01:02 2012 From: techtonik at gmail.com (anatoly techtonik) Date: Tue, 10 Jul 2012 10:01:02 +0300 Subject: [Python-ideas] Python as a tool to download stuff for bootstrapping In-Reply-To: References: Message-ID: On Thu, Jul 5, 2012 at 11:24 PM, Amaury Forgeot d'Arc wrote: > 2012/7/5 anatoly techtonik : >> This makes me kind of sad. You have Python installed. Why can't you >> just crossplatformly do: >> >> mkdir nacl >> cd nacl >> python -m urllib get >> http://commondatastorage.googleapis.com/nativeclient-mirror/nacl/nacl_sdk/update_sdk.py >> python update_sdk.py > > I'm sure there is already a way with standard python tools. Something > along these lines: > > python -c "from urllib.request import urlretrieve; urlretrieve('URL', > 'update_sdk.zip')" > python -m update_sdk.zip > > The second command will work if the zip file has a __main__.py. > Do you think we need other tools? Wow! Nice, but still a hack. I doubt many people wouls accept magic files messing in the root dir of the package. The only way it will look good is: python -m update_sdk.zip sdk_update[.py] but I don't know if that's supported. -- anatoly t. From amauryfa at gmail.com Tue Jul 10 09:51:25 2012 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Tue, 10 Jul 2012 09:51:25 +0200 Subject: [Python-ideas] Python as a tool to download stuff for bootstrapping In-Reply-To: References: Message-ID: 2012/7/10 anatoly techtonik : > Wow! Nice, but still a hack. I doubt many people wouls accept magic > files messing in the root dir of the package. The only way it will > look good is: > > python -m update_sdk.zip sdk_update[.py] > > but I don't know if that's supported. -m does work with zip files (with a __main__.py file) and command line arguments are passed. Or did you expect another kind of support? -- Amaury Forgeot d'Arc From christopherreay at gmail.com Tue Jul 10 09:58:31 2012 From: christopherreay at gmail.com (Christopher Reay) Date: Tue, 10 Jul 2012 08:58:31 +0100 Subject: [Python-ideas] Explanation on how to search the archives In-Reply-To: <4FF887A7.5070101@pearwood.info> References: <4FF887A7.5070101@pearwood.info> Message-ID: Im thinking something like: Python-ideas mailing list Python-ideas at python.org http://mail.python.org/**mailman/listinfo/python-ideas How to Search the Archives: http://mail.python.org/dontReinventTheWheel Searching the archives is the *first* place you should go when exploring new ideas. They are full of (mostly!) excellent ideas, as well as showing how to explain your idea well, and helping you to not reinvent the wheel... or the tabloid. Please search for relevant keywords around your idea before posting to python-ideas. Thanks! As the footer of the emails. This is a small addition which provides some great meta information to "people with great ideas" and also allows a terse and simple response when a veteran detects a repeated idea. -- Be prepared to have your predictions come true -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue Jul 10 11:11:58 2012 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 10 Jul 2012 19:11:58 +1000 Subject: [Python-ideas] Python as a tool to download stuff for bootstrapping In-Reply-To: References: Message-ID: On Tue, Jul 10, 2012 at 5:51 PM, Amaury Forgeot d'Arc wrote: > 2012/7/10 anatoly techtonik : >> Wow! Nice, but still a hack. I doubt many people wouls accept magic >> files messing in the root dir of the package. The only way it will >> look good is: >> >> python -m update_sdk.zip sdk_update[.py] >> >> but I don't know if that's supported. > > -m does work with zip files (with a __main__.py file) and command line > arguments are passed. > Or did you expect another kind of support? -m doesn't handle zip files - you just execute them directly: python