Mailman 3 November 2013 - Python-ideas

Quick Lambdas and Partials (Re: A suggestion for Python 3 vs Python 2)
by Andrew Barnert Nov. 14, 2013

Nov. 14, 2013

I think the discussions on expression-template quick lambdas and auto-partialing functions are both way off topic for the thread about changing Python syntax to allow function calls without parens (or just print, or whatever). And I don't think they really belong on python-ideas at all (unless someone has a suggestion for language or stdlib support for either, which seems unlikely). Since I started both ideas, the derailing is entirely my fault. Anyway, if anyone's interested, I slapped … [View More]

2 1

where statement in Pyret
by Tarek Ziadé Nov. 14, 2013

Nov. 14, 2013

Hey I've read about Pyret on hackernews: http://www.pyret.org/ and found the 'where' statement very compeling. Functions can end with a where that contains small unit tests. >From the documentation example: fun sum(l): cases(List) l: | empty => 0 | link(first, rest) => first + sum(rest) end where: sum([]) is 0 sum([1, 2, 3]) is 6 end It's quite similar to the doctests ideas I guess - but not intended to be documentation like them. I ended up disliking docttests … [View More]

9 16

Stdlib YAML evolution (Was: PEP 426, YAML in the stdlib and implementation discovery)
by anatoly techtonik Nov. 14, 2013

Nov. 14, 2013

FWIW, I am +1 on for the ability to read YAML based configs Python without dependencies, but waiting for several years is hard. Maybe try an alternative data driven development process as opposed to traditional PEP based all-or-nothing style to speed up the process? It is possible to make users happy incrementally and keep development fun without sacrificing too much on the Zen side. If open source is about scratching your own itches, then the most effective way to implement a spec would be to … [View More]allow people add support for their own flavor without disrupting works of the others. For some reason I think most people don't need full YAML speccy, especially if final implementation will be slow and heavy. So instead of: import yaml I'd start with something more generic and primitive: from datatrans import yamlish Where `datatrans` is data transformation framework taking care of usual text parsing (data partitioning), partition mapping (structure transformation) and conversion (binary to string etc.) trying to be as fast and lightweight as possible, bringing vast field for future optimizations on algorithmic level. `yamlish` is an implementation which is not vastly optimized (datatrans to yamlish is like RPython to PyPy) and can be easily extended to cover more YAML (hopefully). Therefore the name - it doesn't pretend to parse YAML - it parses some supported subset, which is improved over time by different parties (if datatrans is done right to provide readable (maintainable + extensible) code for implementation). There is an exisiting package called `yamlish` on PyPI - I am not talking about it - it is PyYAML based, which is not an option for now as I see it. So I stole its name. Sorry. This PyPI package was used to parse TAP format, which is again, a subset. Subset.. It appears that YAML is good for humans for its subsets. It leaves an impression (maybe it's just an illusion) that development work for subset support can also be partitioned. If `datatrans` "done right" is possible, it will allow incremental addition of new YAML features as the need for them arises (new data examples are added). Or it can help to build parsers for YAML subsets that are intentionally limited to make them performance efficient. Because `datatrans` is a package isolating parsing, mapping and conversion parts of the process to make it modular and extensible, it can serve as a reference point for various kinds of (scientific) papers including the ones that prove that such data transformation framework is impossible. As for `yamlish` submodule, the first important paper covering it will be a reference table matrix of supported features. While it all sounds too complicated, driving development by data and real short term user needs (as opposed to designing everything upfront) will make the process more attractive. In data driven development, there are not many things that can break - you either continue parsing previous data or not. The output from the parsing process may change over time, but it may be controlled by configuring the last step of data transformation phase. `Parsing AppEngine config file` or `reading package meta data` are good starting points. Once package meta data subset is parsed, it is done and won't break. The implementation for meta data parsing may mature in distutils package, for AppEngine in its SDK, and merged in either of those, sending patches for `datastrans` to stdlib. The question is only to design output format for the parse stage. I am not sure everything should be convertible into Python objects using the "best fit" technique. I will be pretty comfortable if target format will not be native Python objects at all. More than that - I will even insist to avoid converting to native Python object from the start. The ideal output for the first version should be generic tree structure with defined names for YAML elements. The tree that can be represented as XML where these names are tags. The tree can be therefore traversed and selected using XPath/JQuery syntax. It will take several years for implementation to mature and in the end there will be a plenty of backward compatibility matters with the API, formatting and serializing. So the first thing I'd do is [abandon serialization]. From the point of view of my self-proclaimed data transformation theory, the input and output formats are data. If output format is not human readable - as some linked Python data structures in memory - it wastes time and hinders development. Serializing Python is a problem of different level, which is an example of binary, abstract, memory-only output format - a lot of properties that you don't want to deal with while working with data. To summarize: 1. full spec support is no goal 2. data driven (using real world examples/stories) 3. incremental (one example/story at a time) 4. research based (beautiful ideas vs ugly real world limitations) 5. maintainable (code is easy to read and understand the structure) 6. extensible (easy to find out the place to be modified) 7. core "generic tree" data structure as an intermediate format and "yaml tree" data structure as a final format from parsing process P.S. I am willing to wok on this "data transformation theory" stuff and prototype implementation, because it is generally useful in many areas. But I need support. -- anatoly t. [View Less]

9 12

Issues with inspect.Parameter
by Antony Lee Nov. 13, 2013

Nov. 13, 2013

The docstring of inspect.Parameter indicates the "default" and "annotation" attributes are not set if the parameter does not have, respectively, a default value and an annotation, and that the "kind" attribute is a string. But in fact, the "default" and "annotation" attributes are set to "inspect._empty (== Parameter.empty)" in that case, and the "kind" attribute has type "_ParameterKind" (essentially a hand-written equivalent of IntEnum). I suggest to correct the docstring accordingly, and to … [View More]

2 3

Re: [Python-ideas] A suggestion for Python 3 vs Python 2
by Xuancong Wang Nov. 13, 2013

Nov. 13, 2013

>You probably ought to weigh the ease of typing against readability. Which is more important? By how much? As people are used to python2, I don't think changing print to print() will improve code readability by how much, especially now VIM already highlights the print command in python. Probably you don't use print in python quite often, so you may not sense the difference in typing brackets. If you ever used C/C++/perl frequently, you should sense the inconvenience caused by brackets in … [View More]loop structures. >Separately, have you measured typing speed rigorously? The typing effort varies from person to person, from beginners to experts, maybe you are too expert to sense the difference which is sensitive to beginners. There's no rule-of-thumb and it's not just a speed issue. It's also related to energy spent by your fingers. So obviously, pressing 2 keys, you need to spend at least twice the amount of energy. And even more if you need to hold down 1 key. Anyway, I am just introducing an alternative form to the print() function. As long as this does not reduce the performance of the interpreter significantly, it should not harm. And it should also improve the downward compatibility. On Tue, Nov 12, 2013 at 1:27 PM, Guido van Rossum <guido(a)python.org> wrote: > You probably ought to weigh the ease of typing against readability. Which > is more important? By how much? If 10% more effort typing makes for 5% > better readability, but the code is read 5 times as often as it is edited, > is that then worth it? > > Separately, have you measured typing speed rigorously? > > On Monday, November 11, 2013, Xuancong Wang wrote: > >> >An interesting methodology; however, I think PERL already has >> >conquered this corner of the language world. >> >> I think Perl has much lower efficiency than python, especially you need >> to type a $ before every variable. The input effort of $ is very high >> because you need to press shift. >> >> Also, you need to type {} for every function/for/while structure. >> >> In terms of language efficiency, I think Perl is no comparison to Python. >> >> We can roughly estimate the input effort of every key in the following >> way: >> Normal alphabet keys: effort=1 >> Numbers 0~9: effort=1.2 >> Shift/Tab: effort=0.6 >> Ctrl/Alt: effort=0.8 >> {}[];'\,./-=: effort=1.2 >> (effort measures how difficult it is to press the key) >> Therefore, any composed keys like shift+9='(', the input effort is >> 0.6+1.2=1.8 >> that's why we should try to avoid composed keys if it's not necessary. >> >> >>"Advantages of print being a function: >> * You can override it. Can't do that with a special language element. >> * It can be used in map, lambda, and other expression contexts. >> * Precedence etc follows the normal rules of functions - the arguments >> are all tidily enclosed. >> * Keyword arguments, rather than magical syntax, handle the oddities >> like end=" ". >> * You can easily alias it: "p = print; p('Hello, world!')" >> >> I do agree that print should remain as a function logically. But is there >> a way to make it as simple as in python 2, or even simpler, for example: >> pr >>sys.stderr, 'hello world' >> >> xuancong >> >> >> On Tue, Nov 12, 2013 at 11:56 AM, Mark Janssen <dreamingforward(a)gmail.com >> > wrote: >> >>> On Mon, Nov 11, 2013 at 7:45 PM, Xuancong Wang <xuancong84(a)gmail.com> >>> wrote: >>> > As you know, reading from and writing to IO is a high frequency >>> operation. >>> > By entropy coding theorem (e.g. Huffman coding), an efficient language >>> > should assign shorter language code to more frequent tasks. Typing a >>> '(' >>> > requires holding SHIFT and pressing 9, the input effort is much higher >>> than >>> > that in Python 2. Also, specifying IO has changed from >>* to file=*, >>> which >>> > also becomes more inconvenient. >>> >>> An interesting methodology; however, I think PERL already has >>> conquered this corner of the language world. >>> -- >>> MarkJ >>> Tacoma, Washington >>> >> >> > > -- > --Guido van Rossum (on iPad) > [View Less]

7 7

Extending language syntax
by Gregory Salvan Nov. 13, 2013

Nov. 13, 2013

Hi all, I've just joined you, I hope I'll not make you loose your time with questions already covered. Sorry, if my suggest represents lot of work (certainly too much) and about my lack of knowledge about internal implementation, but I hope opening a discussion can help. I would suggest to introduce a new statement to provide a way to extend language syntax. Abstract: While reading this article about None history<http://python-history.blogspot.fr/2013/11/story-of-none-true-false.html>I … [View More]realised I was not alone to wonder about some aspect of the language. Some of these are: - constant and immutability - null object and "None is a singleton" - readability of code *Constant:* It's a convention, no matter with that. My question is more if there is no memory overuse to have fixed values that behave as variable. *Null Object:* NoneType is not overridable. Having null objects instance of NoneType wouldn't make sense ? Can't assign keywords so "None is a singleton". Wouldn't it be more simple to have "keyword" type and be able to assign a keyword to a variable ? With an object hierarchy like: keyword <- ReservedKeyword I find it can make clearer why some keywords are reserved without changing actual behaviour. *Readability of code:* I have shouldDsl <http://www.should-dsl.info/> use case in mind. The introduction illustrate my thought: "*The goal of Should-DSL is to write should expectations in Python as clear and readable as possible, using an “almost” natural language (limited by some Python language’s constraints).*" I find ShouldDsl perturbing, as the aim is laudable but the implementation stays a hack of the language. I see the influence of functional paradigm in this syntax and agreed with the fact it becomes a "must have". I love Python syntax and find, it has very few limits, but maybe it can be improved by reducing some constraints. One concerns in extending syntax is that the language can evolve on users usage. I see it like an open laboratory where developpers experiments new keywords like "should" implemented in their favorite language and when this keyword is largely accepted by the community it can finally integrate the core. *A solution*: I was first thinking of a statement like "Instance" to declare an immutable object with a single instance (like None), but I then consider how it would be great to be able to extend the syntax. I thought the keyword "Keyword", which exists in clojure for example<http://clojure.org/data_structures#Data%20Structures-Keywords>, can do the job. A first implementation can be: Keyword <name>(<keyword, literal or object>): field = <something> # fixed value: self.field = "something" will raise an error def behaviour(self, *args, **kwargs): # do something def __get__(self, left, right, block_content): # what to do when accessing keyword Inline implementation: Keyword <name>(<keyword, literal or object>) *Examples of use:* *Constant*: Keyword Pi(3.14159) *Null Object* and immutable objects: Class Person(object): def __init__(self, name): self.name = name JohnDoe = Person(name="anonymous") Keyword Anonymous(None, JohnDoe) Anonymous.name = "JohnDoe" # raise an error assert isinstance(Anonymous, type(None)) # is true As Anonymous is immutable , it is also thread safe. The mecanism which consist of freezing an object can be applied in other situation. An interesting approach can be to define keywords identity throught a hash of their value, like value object. *Should keyword*: Keyword ShouldEqual(keyword): def __get__(self, left, right, block): if left != right: raise AssertError() ### weird examples: *Multiline Lambdas*: Keyword MultilineLambda(keyword): def __get__(self, left, right, block_content): def multiline_lambda(*args, **kwargs): self.assert_args_valid(right, args, kwargs) # compare "right" (tuple) with args and raise exception in case not valid return self.run(block_content, args, kwargs) # "run" is a shortcut because I don't know which type is the most appropriate for block content return multiline_lambda divide = MultilineLambda a, b: if b == 0: raise MyDivideByZeroError() return a/b *Reimplementing else*: Keyword otherwise(keyword): def __get__(self, left, right, block): if left: return # left is the result of "if" block self.run(block) if something: # do something otherwise: # other Whereas this example is weird, it seems to me that we can better figure out what language do. Thanks all to have kept python open and give us opportunities to freely discuss about its future implementation. Have a nice day, Grégory [View Less]

6 18

Re: [Python-ideas] A suggestion for Python 3 vs Python 2
by Xuancong Wang Nov. 12, 2013

Nov. 12, 2013

>An interesting methodology; however, I think PERL already has >conquered this corner of the language world. I think Perl has much lower efficiency than python, especially you need to type a $ before every variable. The input effort of $ is very high because you need to press shift. Also, you need to type {} for every function/for/while structure. In terms of language efficiency, I think Perl is no comparison to Python. We can roughly estimate the input effort of every key in the … [View More]

7 7

Re: [Python-ideas] where statement in Pyret
by Skip Montanaro Nov. 10, 2013

Nov. 10, 2013

> Sadly, I *really* dislike that. To me, "where" has absolutely nothing to > do with testing. I see that Pyret also includes a "check" keyword which > also does testing. That seems like a more sensible keyword. "where" seems like it would implement argument constraints. I don't think you want the (necessary) mess of comprehensive unit test code embedded in the source, so nix on the "check" keyword too. It would just obscure the structure of the code. Skip

1 0

Possible new slice behaviour? Was ( Negative slice discussion.)
by Ron Adam Nov. 5, 2013

Nov. 5, 2013

This is one solution to what we can do to make slices both easier to understand and work in a much more consistent and flexible way. This matches the slice semantics that Guido likes. (possibly for python 4.) The ability to pass callables along with the slice creates an easy and clean way to add new indexing modes. (As Nick suggested.) Overall, this makes everything simpler and easier to do. :-) Cheers, Ron """ An improved slice implementation. Even the C source says:… [View More] "It's harder to get right than you might think." both in code and in understanding This requires changing slice indexing so that the following relationships are true. s[i:j:k] == s[i:j][k] both in code and in understanding s[i:j:-1] == s[i:j:1][::-1] And it also adds the ability to apply callables to slices and index's using the existing slice syntax. These alterations would need to be made to the __getitem__ and __setitem__, methods of built-in types. Possibly in Python 4.0. *(I was not able to get a clean version of this behaviour with the existing slice semantics. But the slice and index behaviour of this implementation is much simpler and makes using callables to adjust index's very easy. That seems like a good indication that changing slices to match the above relationships is worth doing. It may be possible to get the current behaviour by applying a callable to the slice like the open, closed, and ones index examples below. ) """ # A string sub-class for testing. class Str(str): def _fix_slice_indexes(self, slc): # Replace Nones and check step value. if isinstance(slc, int): return slc i, j, k = slc.start, slc.stop, slc.step if k == 0: raise ValueError("slice step cannot be zero") if i == None: i = 0 if j == None: j = len(self) if k == None: k = 1 return slice(i, j, k) def __getitem__(self, args): """ Gets a item from a string with either an index, or slice. Apply any callables to the slice if they are pressent. Valid inputes... i (i, callables ...) slice() (slice(), callables ...) """ # Apply callables if any. if isinstance(args, tuple): slc, *callables = args slc = self._fix_slice_indexes(slc) for fn in callables: slc = fn(self, slc) else: slc = self._fix_slice_indexes(args) # Just an index if isinstance(slc, int): return str.__getitem__(self, slc) # Handle slice. rval = [] i, j, k = slc.start, slc.stop, slc.step ix = i if k > 0 else j-1 while i <= ix < j: rval.append(str.__getitem__(self, ix)) ix += k return type(self)('').join(rval) """ These end with 'i' to indicate they make index adjustments, and also to make them less likely to clash with other functions. Some of these are so simple, you'd probably just adjust the index directly, ie.. reversei. But they make good examples of what is possible. And possible There are other uses as well. Because they are just objects passed in, the names aren't important. They can be called anything and still work, and the programmer is free to create new alternatives. """ def reversei(obj, slc): """Return a new slice with reversed step.""" if isinstance(slc, slice): i, j, k = slc.start, slc.stop, slc.step return slice(i, j, -k) return slc def trimi(obj, slc): """Trim left and right so an IndexError is not produced.""" if isinstance(slc, slice): ln = len(obj) i, j, k = slc.start, slc.stop, slc.step if i<0: i = 0 if j>ln: j = ln return slice(i, j, k) return slc def openi(obj, slc): """Open interval - Does not include end points.""" if isinstance(slc, slice): i, j, k = slc.start, slc.stop, slc.step return slice(i+1, j, k) return slc def closedi(obj, slc): """Closed interval - Includes end points.""" if isinstance(slc, slice): i, j, k = slc.start, slc.stop, slc.step return slice(i, j+1, k) return slc def onei(obj, slc): """First element is 1 instead of zero.""" if isinstance(slc, slice): i, j, k = slc.start, slc.stop, slc.step return slice(i-1, j-1, k) return slc - 1 def _test_cases1(): """ # test string >>> s = Str('0123456789') # |0|1|2|3|4|5|6|7|8|9| # 0 1 2 3 4 5 6 7 8 9 10 # 10 9 8 7 6 5 4 3 2 1 0 >>> s[:] '0123456789' >>> s[:, trimi] '0123456789' >>> s[:, reversei] '9876543210' >>> s[:, reversei, trimi] '9876543210' >>> s[::, trimi, reversei] '9876543210' # Right side bigger than len(s) >>> s[:100] Traceback (most recent call last): IndexError: string index out of range >>> s[:100, trimi] '0123456789' >>> s[:100, trimi, reversei] '9876543210' >>> s[:100, reversei] Traceback (most recent call last): IndexError: string index out of range >>> s[:100, reversei, trimi] '9876543210' # Left side smaller than 0. >>> s[-100:] Traceback (most recent call last): IndexError: string index out of range >>> s[-100:, trimi] '0123456789' >>> s[-100:, trimi, reversei] '9876543210' # Slice bigger than s. >>> s[-100:100] Traceback (most recent call last): IndexError: string index out of range >>> s[-100:100, trimi] '0123456789' # Slice smaller than s. >>> s[3:7] '3456' >>> s[3:7, reversei] '6543' # From left With negative step. >>> s[::-1] '9876543210' >>> s[::-1, reversei] '0123456789' >>> s[:100:-1, trimi] # j past right side '9876543210' >>> s[-100::-1, trimi] # i before left side '9876543210' >>> s[-100:100:-1, trimi, reversei] # slice is bigger '0123456789' # Null results >>> s[7:3:1, trimi] '' >>> s[7:3:-1, trimi] '' # Check None values. >>> s[:] '0123456789' >>> s[None:None] '0123456789' >>> s[None:None:None] '0123456789' >>> s[:: 1] '0123456789' >>> s[::-1] '9876543210' >>> s[None:None:1] '0123456789' >>> s[None:None:-1] '9876543210' # Check error messages. >>> s[0:0:0:0] Traceback (most recent call last): SyntaxError: invalid syntax >>> s[5:5:0] Traceback (most recent call last): ValueError: slice step cannot be zero # And various other combinations. >>> s = Str('123456789') >>> s[3, onei] '3' >>> s[4:8, onei] '4567' >>> s[4:8, onei, openi] '567' >>> s[4:8, onei, closedi] '45678' """ def _test(): import doctest print(doctest.testmod(verbose=False)) if __name__=="__main__": _test() [View Less]

4 7

requests in the stdlib?
by Matěj Cepl Nov. 4, 2013

Nov. 4, 2013

Afternoon (or whatever else), gentlemen and gentle ladies! I guess I am not the first one who experienced a profound life-changing epiphany of meeting with the requests package. Yesterday after fighting for hours with urllib2 (and GitHub API) and failing, I have succeeded with requests in five minutes. Despite how much I dislike using libraries outside of the stdlib (yes, unittest is the way to go, please, no nosetests and py.test for me), I am now persuaded that requests API is the … [View More]

6 6