Python-ideas February 2014

python-ideas@python.org

98 participants
73 discussions

code segments
by spir 20 Feb '14

20 Feb '14

I often find myself needing to wrap little pieces of code into a whole "object" (programming element or value), sometimes with a name. I feel the same kind of lack as an FP programmer having to code in a language without function "objects"; and indeed the situation is very similar. The cases where i need that in fact follow a schema, let's call it "event". In games, an event is usually something that happens in a particular situation: there is a logic like situation --> happening, an event is in fact the binding of 2 elements. We find something similar with automation (which also is event-driven) sequences: a tree of stages which are reached under a condition and each command a given action: condition --> action. More generally, one may find a kind of cause --> effect schema. An event can be conceptualised as a {cause effect} pair. But what are the pair's elements? And how to encode them in a program? monster'appears : Event{ cause : and monster.is'hidden (= character.pos (33 99)) effect : monster.move 37 101 monster.groar 'fiercefully monster.is'hidden :: false character.scream } The cause is conceptually a logical expression; but it can be arbitrarily complex --thus may require multiple statements if only for readability, also for debugging or other "meta" needs. Otherwise, it is like a proper function, with an output (here a logical value) and no effect. But it does not take any input! instead, any pieces of data it uses are present in the surrounding scope: they _must_ be there, if i may say by logical necessity. The effect is an action, like a procedure that changes the world and computes no product. Similarly, it takes no input but finds its data in the outer scope. Thus, we have 2 kinds of input-less procedures. Otherwise, the notion is somewhat like Ruby blocks I guess. This is also similar to the recently proposed "inline" functions on python-ideas (reason why cc to this list). There may also be a relation to dynamic scoping, since such code segments in fact appear to take their input from the caller's scope: but it is not a _caller_, instead a kind of surrounding block and scope, like in case of inlining. I think _this_ kind of semantics, similar to inlining, is the actual value of dynamic scoping, and what we may miss with static scoping only and ordinary procedures only. We may need a way to have "reified" blocks of code executed in the surrounding scope as if they were explicitely written there, or inlined (or C-like macros). Actual functions/procedures would be overkill, since there is no stack frame (or similar), thus all the typical "prologue" and "epilogue" of procedure calls is unneeded. We just need to jump there, wherever the code lies in memory, and come back. Or, inline à la C, or like C macros, but this in my view does not match the semantics. What I'm considering is code segment "objects" like procedures but without: 1. input variables, 2. local scope, 3. call/return [0], three related points in fact. Even more generally, one may note ordinary "functions", in mainstream langs and FP alike, combine a number of properties, the ones listed above plus: 0. more basically, general notion of a block of code as programming element 4. in addition, possible output product, for a function-like code segment (maybe I miss some) For the present notion of code segments, all i need is 0, plus 4 in some cases. [2] I'm not sure at all this is a good idea, however --but I like it and would regularly use it if available. d [0] With register save, new stack frame construction, param passing (prologue), etc, and undoing of all that. [1] A static location would not be thread safe; the only other possibility I can think of is dynamic allocation on the heap: overkill. A real alternative is _not_ passing the product back, but noting that at a low level, there are no functions, instead only actions that write somewhere. Eg: x := DIV y z actually means: DIV x (y z) (and in fact at an even lower level x is y, so we get "DIV x z") Similarly, when part of a higher expression we need temp storage location, eg: a := SUM b (DIV c d) requires: DIV temp (c d) SUM a (b temp) Also: IF (EQUAL a b) THEN do-that gives: EQUAL temp (a b) IF temp THEN do-that Function-like code segments would take the place of explicit expressions here. For instance: a := segment-name a := SUM b segment-name if segment-name then do-that Thus, a rewriting rule can get read of their output altogether, I guess. [2] Detail: in a low-level language, it may be easy to do that: a pair of goto; but the instruction for jumping back must accept a _variable_ target, and the the forward one may also be variable at times. In assembly, it's jumping to given addresses, or even just setting the PC register (program counter); there is no issue of variable targets at this level, they're variable adresses in general, meaning pointers. The only issue is scope restriction on such jumps (whether and how to possibly jump into other memory segments). A final point is the output of function-like code segments: we may need a single, temporary stack slot for it [1]. All these issues are solved for ordinary procedures.

3 2

for-while statement
by Alejandro López Correa 19 Feb '14

19 Feb '14

Hello, I think adding an optional "WHILE" clause in "FOR" loops might be useful sometimes (shorter code and/or improved readability): for #VAR# in #SEQUENCE# while #WATCHDOG_EXPRESSION#: #CODE_BLOCK# Examples: keepRunning = True for i in range(100) while keepRunning: keepRunning = do_some_work( i ) found = False for candidate in sequence while not found: try: process( candidate ) found = True except InvalidCandidate: pass retryCount = 7 for i in range(1,1+retryCount) while resource.acquire() == FAIL: sleep( i**2 ) At the moment, I usually implement this either with ugly breaks: for i in range(100): if not do_some_work( i ): break found = False for candidate in sequence: try: process_candidate() except InvalidCandidate: pass else: found = True break Or with while loops and counters (or counting-like expressions): i = 1 while i <= retryCount and not resource.acquired: if resource.acquire() == FAIL: sleep( i**2 ) i += 1 Of course, actual code tends to be more complex, with "keepRunning" being modified in some branches of "IF" blocks, and there might be nested loops where the exit condition for the outer one is set in the inner loop. Compare these two examples: found = False for filePath in glob( '*.data' ): for line in open( filePath, 'rt' ): if line.startswith( '#' ): continue if handle( line ): found = True break if found: break found = False for filePath in glob( '*.data' ) while not found: for line in open( filePath, 'rt' ) while not found: if line.startswith( '#' ): continue if handle( line ): found = True -- Alejandro

6 6

os.path.here()
by Giampaolo Rodola' 19 Feb '14

19 Feb '14

Hello python devs. One thing which appears to be pretty common in many scripts is figuring out module's parent directory path. I often find myself writing: HERE = os.path.abspath(os.path.dirname(__file__)) ...at the top of a module. Also, it is not rare to do further concatenations in order to declare where static files are located: CONF_FILE = os.path.abspath(os.path.join(HERE, 'app.conf')) CERT_FILE = os.path.abspath(os.path.join(HERE, 'cert.pem')) THIS_FILE = os.path.abspath(os.path.join(HERE, __file__)) A quick search within Python source code shows this is indeed a very common pattern: $ find . -name \*.py | xargs grep "os\.path" | grep __file__ Just some examples (there are *many*): http://hg.python.org/cpython/file/186f6f56f4bc/Lib/distutils/tests/__init__… http://hg.python.org/cpython/file/186f6f56f4bc/PCbuild/build_tkinter.py#l11 http://hg.python.org/cpython/file/186f6f56f4bc/PCbuild/build_ssl.py#l68 http://hg.python.org/cpython/file/186f6f56f4bc/Tools/msi/uisample.py#l2 http://hg.python.org/cpython/file/186f6f56f4bc/Lib/distutils/tests/test_con… http://hg.python.org/cpython/file/186f6f56f4bc/Lib/unittest/test/testmock/_… http://hg.python.org/cpython/file/186f6f56f4bc/Lib/pydoc.py#l2463 I think there's space for improvements here so my proposal is to add a simple os.path.here() function working like this: >>> # assuming /home/giampaolo/app/run.py is the current module: >>> os.path.here() /home/giampaolo/app/ >>> os.path.here(__file__) /home/giampaolo/app/run.py >>> os.path.here('..') /home/giampaolo/ The implementation is pretty straightforward: def here(concat=None): """Return the absolute path of the parent directory where the script is defined. """ here = os.path.abspath(os.path.dirname(__file__)) if concat is not None: here = os.path.abspath(os.path.join(here, concat)) return here Thoughts? -- Giampaolo - http://grodola.blogspot.com

3 4

An Everything singleton
by Serhiy Storchaka 19 Feb '14

19 Feb '14

This crazy idea is inspired by a discussing in Python-Dev which should be here. Currently we have a None singleton which representing nothing and raises an exception in almost all operations (except equality, etc). What about introducing its antonym, an Everything singleton (the name is discussable), which newer raises an exception in any operation, but returns consistent value? >>> Everything + 2 2 >>> Everything < 3 True >>> Everything * 'foo' 'foo'

10 9

Re: [Python-ideas] Inline Function - idea
by Alex Rodrigues 18 Feb '14

18 Feb '14

> Alex, can you explain the difference (if any) between your proposal and dynamic scoping? > > -- Steven Inline functions would be a bit like dynamic scoping and a bit like macros. The main difference from dynamic scoping is that they would not search up beyond their parent to find names, since they act exactly like code injected at the spot they are called it is expected that the variables they are using (like the ones in it's parent) are either locals or globals. I'm not sure that that has many advantages outside of demanding less from the runtime, but that's how I imagined it. IMO The biggest advantage to inline functions over other constructs is ease of understanding. We may you be used to understanding scoping after lots of programming, but it's not always intuitive. On the other hand it is extremely intuitive to understand "when you call this, all that code runs exactly as if you had typed it here". - Alex

4 3

Re: [Python-ideas] pickling obscured function references
by Henry Harrison 18 Feb '14

18 Feb '14

Apologies. Yes, I had in mind something in-between a proposal and a question, which in retrospect was not the way to go about it. And I was not aware of python-list, so apologies again for not researching the options for having this discussion. In any case after sleeping on it I'm pretty sure the answer to my question is "because backwards compatibility" which means that the proposal would be moot. Sorry again for wasting your time. - Henry > On Feb 18, 2014, at 8:46, Terry Reedy <tjreedy(a)udel.edu> wrote: > > > On 2/18/2014 1:41 AM, Henry Harrison wrote: > > > >> I can't be the first to have thought of this, so there must be a reason > >> this isn't the case, but I have to ask. Why is __main__ the fallback > >> when pickle can't find a function reference? > >> > >> Instead of something like: > >> os.path.basename(inspect.getsourcefile(func))[:-3] > >> > >> Thanks for humoring my curiosity, > > > > A place to ask questions and 'humor curiosity' is python-list. > Python-ideas is for ideas for improving future version of python. > > I think he may have actually wanted to propose this change, but posted it > in an overly tentative way so that if it was a bad idea (as, after all, > most language change proposals are...), people would explain gently why > it's a bad idea instead of just rejecting it out of hand. (I don't think > it's a good idea to post that way on this list--in fact, it's likely to > have the opposite of the intended effect--but I can understand why someone > might expect it to be.) >

2 1

pickling obscured function references
by Henry Harrison 18 Feb '14

18 Feb '14

Hello all, I can't be the first to have thought of this, so there must be a reason this isn't the case, but I have to ask. Why is __main__ the fallback when pickle can't find a function reference? Instead of something like: os.path.basename(inspect.getsourcefile(func))[:-3] Thanks for humoring my curiosity, - Henry

3 2

if expensive_computation() as x:
by Ram Rachum 18 Feb '14

18 Feb '14

Hi everybody, Please excuse the recent torrent of crazy ideas :) I was reading code of the Shpaml project, trying to make a patch, while I saw code that looked inelegant to me. I considered how to improve it, but then saw I don't know how to. The code paraphrased is this: if expensive_computation_0(): x = expensive_computation_0() # Do something with x... elif expensive_computation_1(): x = expensive_computation_1() # Do something with x... elif expensive_computation_2(): x = expensive_computation_2() # Do something with x... The problem here is that we're doing the expensive computation twice. That's because we first need to check its truth value, and if it's true we need to actually use the value and do stuff with it. One solution would be to calculate it and put it in a variable before checking truth value. The problem with that is, besides being quite verbose, it doesn't work with the `elif` lines. You want to calculate the values only if the `elif` branch is being taken. (i.e. you can't do it before the `if` because then you might be calculating it needlessly since maybe the first condition would be true.) Obviously this can be "solved", i.e. rewritten in a way that wouldn't call the expensive operation twice, but the solution would probably be quite verbose, which is something we don't want. My suggestion: if expensive_computation_0() as x: # Do something with x... elif expensive_computation_1() as x: # Do something with x... elif expensive_computation_2() as x: # Do something with x... If you'd like to bind to a variable only a part of the condition, this would work too: if x<5 with expensive_computation_0() as x: # Do something with x What do you think? Ram.

13 28

docs.python.org: Short URLs
by Markus Unterwaditzer 17 Feb '14

17 Feb '14

http://cryto.net/~joepie91/blog/2013/02/19/the-python-documentation-is-bad-… A few of you might have read that infamous rant about Python's community and documentation and Brian Curtins response: http://blog.briancurtin.com/posts/why-i-dont-feel-so-bad.html Like Brian, i think that the unconstructive tone ruined the points the author was trying to make, a part of which i actually agree with. So, here is an attempt at a positive consequence of this rant. The author talks about the inaccessibility of Python's documentation via Google compared to PHP's. One can easily verify that by themselves: Just enter the name of any builtin into the search engine at docs.python.org, such as str, float, or print. In none of these cases, the appropriate documentation is the first result. The search engine at php.net is, on the other hand, a breeze to use, probably because the search engine doesn't have to take namespaces into account when searching. The suggestion i want to make isn't just "make the search engine better". I know that such things are better filed against Sphinx. What i actually wanted to suggest is a convenience feature that is available at php.net: http://php.net/str_replace directly shows the documentation of PHP's str_replace. Concretely, I propose that, for example, http://docs.python.org/2/str redirects to the documentation for `str` in Python 2 , while http://docs.python.org/3/str, http://docs.python.org/str and/or http://python.org/str redirect to the documentation for `str` in Python 3. Here is a simple script that roughly shows how i would expect this redirect to resolve the given object names: import sphinx.ext.intersphinx as intersphinx class FakeSphinxApp(object): def warn(self, x): print(x) baseurl = 'http://docs.python.org/3/' inv = intersphinx.fetch_inventory(FakeSphinxApp(), baseurl, baseurl + 'objects.inv') index = {} for value in inv.values(): index.update(value) def url_for_object(s): return index[s][2] # >>> url_for_object('str') # 'http://docs.python.org/3/library/stdtypes.html#str' # >>> url_for_object('datetime.datetime') # 'http://docs.python.org/3/library/datetime.html#datetime.datetime' I am aware that the pydoc command-line utility ships with the default distribution of Python. However, i don't think any beginner wants to get in touch with the command line more than what is absolutely neccessary, and i also doubt that people are aware of this utility. I personally also haven't used pydoc and Python's `help` builtin mostly because of this: $ pydoc datetime.datetime Help on class datetime in datetime: datetime.datetime = class datetime(date) | datetime(year, month, day[, hour[, minute[, second[, microsecond[,tzinfo]]]]]) | | The year, month and day arguments are required. tzinfo may be None, or an | instance of a tzinfo subclass. The remaining arguments may be ints. | | Method resolution order: | datetime | date | builtins.object | | Methods defined here: | | __add__(...) | x.__add__(y) <==> x+y | | __eq__(...) | x.__eq__(y) <==> x==y | | __ge__(...) | x.__ge__(y) <==> x>=y | | __getattribute__(...) | x.__getattribute__('name') <==> x.name ... What do you think? -- Markus

3 2

Re: [Python-ideas] docs.python.org: Short URLs
by Markus Unterwaditzer 17 Feb '14

17 Feb '14

I did mean to take it to the list. Just replying so it is there content-wise. On 17 February 2014 15:49:16 CET, Chris Angelico <rosuav(a)gmail.com> wrote: >Not sure if you meant to mail me this privately rather than post >on-list. I'll keep this private for now, but feel free to take it to >the list. > >On Tue, Feb 18, 2014 at 1:42 AM, Markus Unterwaditzer ><markus(a)unterwaditzer.net> wrote: >> On Tue, Feb 18, 2014 at 01:34:25AM +1100, Chris Angelico wrote: >>> I like the idea, but your URLs might conflict with other things. >> >> I'd just display the other thing instead then. Actually i think this >should be >> implemented in the 404 error handler or something like that. > >That could be done, but I was thinking more of a safe way to type in a >URL - maybe with a browser plugin, even - that can't possibly bring up >the "wrong thing". > >>> Would it be too awkward if namespaced? >>> >>> http://docs.python.org/[ver]/kw/str >>> >>> and then have a keyword index (hence "kw", which is nice and short) >>> which could have function/class names, common search keywords, a few >>> aliases (regex -> re), etc, etc. >> >> We can have that too, additionally. /name/ would strike me as more >obvious, >> though this is just a matter of preference. > >Either way. "kw" was the best thing I could come up with at short >notice. My point is to namespace it; what the name actually is can >always be bikeshedded :) > >ChrisA

6 12

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Python-ideas February 2014