Jump to function as an an alternative to call function
-Jumping to a function as opposed to calling a function When a function is jumped to, it inherits the variables in the caller’s local namespace and is free to modify them or add new local variables, unlike a normal function call, wherein the caller’s namespace is inaccesible. At present, the only way I know of to accomplish this is to bundle all variables in the caller method’s namespace as properties of the method’s instance and have the callee method modify those properties. Though it is easier to read code written this way, it resulted in a great deal of redundancy in the code I was writing. The task I was attempting to accomplish was approximately this: class Verb: def __init__(self, parameters): self.parameters = parameters def conjugate(self): #Using the parameters, chose an appropriate list of morphemes to prefix and suffix to the verb self.morphemelist = morphemelist for morpheme in self.morphemelist: morpheme.mutate() returnstring = ‘' for morpheme in self.morphemelist: returnstring = returnstring + morpheme.form returnstring = returnstring class Morpheme: def __init__(self, verb, form, precedingmorpheme, succeedingmorpheme): self.verb = verb self.form = form self.precedingmorpheme = precedingmorpheme self.succeedingmorpheme = succeedingmorpheme def mutate(self): #Using the verb’s parameters and the type and form of the preceding and succeeding morpheme, mutate this morpheme’s form so that #a correct verb form is produced self.form = newform class Ban(Morpheme): def __init__(self, verb, form): super().__init__(verb, ‘ban’) def mutate(self): #This morpheme has mutation logic unique to itself but with many similarities to the default morpheme’s mutation logic self.form = newform Each subclass of Morpheme has its own slightly different mutate method. Some subclasses of Morpheme needed to access and manipulate a great deal of information about the verb and their surroundings, while other subclasses’ mutate methods differed little from the default. Most of the variables manipulated by the mutate method are not used by any other methods of class Morpheme and thus should not be made properties of the attached instance. What I wish I were able to do is write many little methods that modify the same set of local variables and compose them together into a complete mutate method. This would be effectively equivalent to having the “jump to function” calls be replaced with the text in the function’s body, like a C language macrosubstitution, but without using anything like preprocessor directives. Let foo(a, b) be the syntax to call foo(a, b), and foo(a, b)% be the syntax to jump to foo(a, b) def foo(a): return a + c def bar(a, c): return foo(a) def bazz(a, c): return foo(a)% c = 5 call = bar(1, 3) jump = bazz(1, 3) After execution, call in the above code would be 6 and jump in the above code would be 4.
Hi Jacob Thank you for your problem. I'll focus on your simple example (lightly edited for clarity)
This may allow you to solve your verb and morpheme problem. But I hope there's an easier way. One that can be done using Python as it is now. My understanding is that you want a stack. Similar to the Python's function call stack, but with something extra. So that functions execute in an richer context. Here's an idea. Start by giving us a toy example of a calculation you'd like to do. Then the problem is: how to do this in Python. By the way, are there any existing Python libraries for this sort of thing? I hope this helps. -- Jonathan
On Thu, Aug 16, 2018 at 4:52 AM, Jacob Solinsky <jacobsolinsky@gmail.com> wrote:
-Jumping to a function as opposed to calling a function
When a function is jumped to, it inherits the variables in the caller’s local namespace and is free to modify them or add new local variables, unlike a normal function call, wherein the caller’s namespace is inaccesible. At present, the only way I know of to accomplish this is to bundle all variables in the caller method’s namespace as properties of the method’s instance and have the callee method modify those properties. Though it is easier to read code written this way, it resulted in a great deal of redundancy in the code I was writing.
You're trying to shed encapsulation (by having the jumped-to function operate in the "caller's" namespace), but also pass parameters to it. That's going to cause a lot of confusion. I'm sympathetic to the problem, but I don't think functions are the solution here. You want some form of code block. There've been a number of proposals along those lines, and so far, not one of them has been really implementable. Check out some of the prior theories and see if one of them will work for you. ChrisA
Jumping into functions that mutate variables in the calling scope sounds a lot like "GoTo" <https://en.wikipedia.org/wiki/Considered_harmful> which is notorious for leading to code that's very hard to reason about. Your functions would implicitly require that you assign variables in the calling scope before calling the function instead of explicitly requiring them in the function's signature. That can be very confusing. A lot of times, when I find that my function call signatures have become unwieldy, it's a sign that I need to bundle a lot of related variables into an object and either define methods on that object or pass that object to functions instead of the large number of variables that it encapsulates. You just use 'self' as a sort-of back-pack that carries around what you need. It's difficult to see what you're after from your Verb and Morpheme example. They don't seem to call any functions that use variables from the caller's scope. Can you flesh that out a bit better? On Wed, Aug 15, 2018 at 3:24 PM, Chris Angelico <rosuav@gmail.com> wrote:
On Thu, Aug 16, 2018 at 7:40 AM, Abe Dillon <abedillon@gmail.com> wrote:
Drop the word "function" and it's no longer about a goto - it's about a block of code that can be deposited into another context. If you were to copy and paste that code, its meaning would be defined by where it is used; the idea of a "code block" is that you can do that with something that can be passed around like a function, but isn't a function. We could borrow a term from the Python grammar and call it a "suite", perhaps, but "code block" is probably the best term. As mentioned, there have been previous proposals along these lines. By the way:
It's difficult to see what you're after from your Verb and Morpheme example.
You're responding to MY post, which is a response to Jacob's. Please watch your use of pronouns; it would help a lot if you weren't top posting, and were directly responding to specific people's specific comments. ChrisA
[Chris Angelico]
Drop the word "function" and it's no longer about a goto - it's about a block of code that can be deposited into another context.
The only context I've used goto is in a TI-83 scientific calculator, but in that, it was just about jumping around to different labels in code and had nothing to do with functions. Though this may not have been the "canonical" goto that Dijkstra considered harmful. [Chris Angelico]
By the way:
It's difficult to see what you're after from your Verb and Morpheme example.
Sorry, I ran into a lot of problems trying to use the Google Groups interface to post, so I've resorted to strictly using email. Unfortunately, that makes the structure of the conversation less obvious to me. I'll try to be more mindful of this in the future. On Wed, Aug 15, 2018 at 4:58 PM, Chris Angelico <rosuav@gmail.com> wrote:
What I had hoped to do was use a preamble code block to collect all of the most common queries called by the mutate function in the local namespace, for example C = 'bpgkdtszSZjCmnywh' M = 'mn' class Morpheme: #stuff def preamble(self): ps = self.precedingmorpheme.form ss = self.succeedingmorpheme.form ssc = re.match(f'^[{C}]', self.succedingmorpheme.form) #Whether or not the following morpheme is consonant initial ssm = re.match(f'[{M}]$', self.precedingmorpheme.form) #Whether or not the preceding morpheme is nasal final these local variables are used quite often in the mutate methods, of which there are several dozen, so storing them by default saves a lot of typing. class Ban(Morpheme): def __init__(self): super().__init__(verb, precedingmorpheme, succeedingmorpheme, form = 'ban') def mutate(self): self.preamble()% if ps[-1] == 'd': self.form = 'p' + self.form[1:] self.preceding.form = '' if ssc: self.form = self.form + 'E' if ssm: self.form = re.sub('an', 'An', self.form) What I actually ended up doing is placing these queries in the Morpheme __init__ method def __init__(self, ...): self.ps = self.precedingmorpheme.form self.ss = self.succeedingmorpheme.form self.ssc = re.match(f'^[{C}]', self.succedingmorpheme.form) #Whether or not the following morpheme is consonant initial self.ssm = re.match(f'[{M}]$', self.precedingmorpheme.form) #Whether or not the preceding morpheme is nasal final This works perfectly fine, but I found cluttering the Morpheme object instances with flags and such to be inelegant, since these flags are only used by the mutate method. Also, without using a hacky solution like making Morpheme a subclass of types.SimpleNamespace, every new flag I might want to inject has to have a default value set in the __init__ method to prevent an AttributeError from being raised. Anyways, I will look around in the mail list for discussions of Code blocks, now that I know they are called. On Wed, 15 Aug 2018, 16:41 Abe Dillon, <abedillon@gmail.com> wrote:
On Thu, Aug 16, 2018 at 8:53 AM, Jacob Solinsky <jacobsolinsky@gmail.com> wrote:
Ahh, I see what you mean. I was thinking the other way - a single mutate function that has a part in there saying "now run the code block for this particular subclass". What would be really nice would be something like: from preamble() import * I'm fairly sure that (per se) isn't going to fly, but it would be interesting as a concept. Would need a new spelling. The quirky part of my brain is trying to figure if class namespace can be abused for this. ChrisA
[Jacob Solinsky]
these local variables are used quite often in the mutate methods, of which there are several dozen, so storing them by default saves a lot of typing.
There are several things you can do to alleviate the typing problem: 1) Get an IDE that has auto-complete. I recommend PyCharm Community Edition, but there are several. 2) Use better naming: there are many shorter synonyms for "preceding" and "succeding" and the word "morpheme" is redundant. 3) define as much as you can in the higher scopes: C = re.compile("^[bpgkdtszSZjCmnywh]").match M = re.compile("^[mn]$").match class Morpheme: @property def pf(self): return self.pre.form @property def sf(self): return self.suc.form @property def precm(self): return C(self.pf) @property def sucmm(self): return M(self.sf) ... I think what would help most of all is reorganizing your code. As I understand it, a morpheme is the "smallest grammatical unit in a language" <https://en.wikipedia.org/wiki/Morpheme> so maybe it makes more sense to make the morpheme class fairly simple and not have a morpheme have any inherent awareness of its preceding or succeeding morphemes. That seems like the job of a larger construct that would deal with sequences of morphemes. The way you have it right now, the morpheme seems to play a double role a linked-list and a gramatical unit. [Jacob Solinsky]
I found cluttering the Morpheme object instances with flags and such to be inelegant, since these flags are only used by the mutate method.
That may be an indicator that you're trying to do too much with the morpheme class. [Jacob Solinsky]
Yes, you typically want to define all your object attributes in __init__ otherwise your objects become fragile and can break if you don't call the right methods in the right order. It's generally considered a bad idea to define instance attributes anywhere else unless it's something like a "private" helper method that gets called inside __init__. [Jacob Solinsky]
Anyways, I will look around in the mail list for discussions of Code blocks, now that I know they are called.
Don't take this the wrong way, but it might be better to acquaint yourself with the set of tools already available before looking for new ones.
Jacob Solinsky writes:
What you're asking for is basically a "Ruby block", AFAICS. This has been requested many times, but most of the senior Pythonistas consider that it leads to a very high risk of unreadable and/or buggy code at no gain in power (there are several ways to efficiently accomplish what you want to do, you just don't like them[1]), and few cases where it's more readable/less buggy. I think if you search "ruby block site:mail.python.org" you'll probably bring up the threads. You won't find a lot of encouraging discussion, I'm afraid. What I would do in your case is class Morpheme: class Preamble: def __init__(self, m): # "m" for "morpheme" self.ps = m.precedingmorpheme.form self.ss = m.succeedingmorpheme.form self.ssc = re.match(f'^[{C}]', m.succedingmorpheme.form) self.ssm = re.match(f'[{M}]$', m.precedingmorpheme.form) def mutate(): p = self.Preamble(self) # use p.ps, p.ss, etc It's a little less efficient, of course, because of overhead managing the Preamble objects. Another possibility would be to use properties, so that instead of *storing* these temporary variables on the Morpheme, they would be computed on the fly: class Morpheme: @property def ps(self): return self.precedingmorpheme.form @property def ss(self): return self.succeedingmorpheme.form @property def ssc(self): return re.match(f'^[{C}]', self.succeedingmorpheme.form) @property def ssm(self): return re.match(f'[{M}]$', self.precedingmorpheme.form) and so on. Then you can access them with "self.ps" etc in the mutate method. This might be considered elegant (although verbose in the Morpheme definition) if these properties are usually used only once per mutate call, and "expensive" properties like ssc and ssm are never called more than once. (If they are called multiple times, it's easy to create a cache.) Of course property accesses do add method call overhead, but they look nicer than method call syntax. On the other hand, if "many" properties are expensive, and "most" mutate methods use only a few of these, you could save runtime this way, since the expensive methods are called only as-needed. I don't understand the "need default value in __init__" issue, since "inject" implied to me that you initialize before use. But if that's a problem for ordinary attributes, I presume it'll be more annoying with properties because you have to write a method for it. So I don't know if this is helpful at all. Footnotes: [1] "I don't like it" is a sufficient reason to propose a change. I'm simply saying you have the *power* to do what you want.
Sorry for the double post, but I wanted to make sure you saw my original misplaced post: Jumping into functions that mutate variables in the calling scope sounds a lot like "GoTo" <https://en.wikipedia.org/wiki/Considered_harmful> which is notorious for leading to code that's very hard to reason about. Your functions would implicitly require that you assign variables in the calling scope before calling the function instead of explicitly requiring them in the function's signature. That can be very confusing. A lot of times, when I find that my function call signatures have become unwieldy, it's a sign that I need to bundle a lot of related variables into an object and either define methods on that object or pass that object to functions instead of the large number of variables that it encapsulates. You just use 'self' as a sort-of back-pack that carries around what you need. It's difficult to see what you're after from your Verb and Morpheme example. They don't seem to call any functions that use variables from the caller's scope. Can you flesh that out a bit more to show where the problem arrises?
Goto considered harmful. https://homepages.cwi.nl/~storm/teaching/reader/Dijkstra68.pdf&ved=2ahUKEwi8lqHYqfDcAhXImOAKHeAzDmsQFjAEegQIBhAB&usg=AOvVaw1CZug_36-PbevItYXTb7SR On Wed, Aug 15, 2018, 3:52 PM Jacob Solinsky <jacobsolinsky@gmail.com> wrote:
Hmm.. link broke. Is this right? https://www.google.com/url?sa=t&source=web&rct=j&url=https://homepages.cwi.nl/~storm/teaching/reader/Dijkstra68.pdf&ved=2ahUKEwi8lqHYqfDcAhXImOAKHeAzDmsQFjAEegQIBhAB&usg=AOvVaw1CZug_36-PbevItYXTb7SR On Wed, Aug 15, 2018, 8:35 PM David Mertz <mertz@gnosis.cx> wrote:
How about just https://homepages.cwi.nl/~storm/teaching/reader/Dijkstra68.pdf On Wed, Aug 15, 2018 at 5:38 PM David Mertz <mertz@gnosis.cx> wrote:
I have no opinion on the proposal, but the equation with goto is dead wrong. Yes, technically this is a goto. But so is a regular function call. Jumping to a function makes thd code _easier_ to reason about, statically, not harder. When you call an arbitrary function, you have no idea whether it will return or not. When you jump to a function, you _know_ it will not return. That's strictly more information, based solely on the syntax of the statement. Yes, if you have local functions mutating the state then jumpung to them makes the code difficult to read. *but so is calling them*, with the addition of the returning that you have to track. Jumping to functions is procedural programming. There is no reasonfor it to be "considered harmful" any more than calling funcions; indeed even less so. Elazar On Wed, Aug 15, 2018, 17:42 Eric Fahlgren <ericfahlgren@gmail.com> wrote:
On Wed, Aug 15, 2018 at 08:35:35PM -0400, David Mertz wrote:
Goto considered harmful.
Fortunately this proposal has nothing to do with goto. Elazar is correct that its a kind of subroutine call, just like an ordinary function call, except the scoping rules are different. And for the record, not everyone agrees that Dijkstra is correct about goto. Certainly unstructured code is harmful, but we use restricted forms of goto all the time, we just don't call it by that name: - loops - continue - break - if...else - function calls - exception handling Just like goto, these are all jumps which change the execution order of your code. And some people defend limited, careful use of explicit goto, including Donald Knuth. -- Steve
When did you last read the Dijkstra paper. It's short and he explains very well why exactly all the constructs you mention are unlike Goto. This isn't a proposal to subroutines, it's a proposal for subroutines in which all lexically scoped variables are implicitly nonlocal. That's basically Goto, certainly course enough to suffer all of Dijkstra's complaints. Yes, technically this could be called dynamic scope instead. But notwithstanding the use in elisp, that's pretty widely regarded as a bad ideas (even by the emacs developers who regret that early decision). On Wed, Aug 15, 2018, 9:41 PM Steven D'Aprano <steve@pearwood.info> wrote:
On Wed, Aug 15, 2018 at 09:58:28PM -0400, David Mertz wrote:
When did you last read the Dijkstra paper. It's short and he explains very well why exactly all the constructs you mention are unlike Goto.
Of course I've read it, and no he does not. He only mentions if...else and looping. He doesn't mention break, continue, exceptions (I don't think exceptions had even been invented in 1968). I think that if Dijkstra were still alive, he would probably chose to understate or dismiss the similarities between goto and break/continue. Others might choose to focus on the similarities rather than the differences, e.g. "continue" is just a goto to the start of the loop, "break" is goto the end of the loop. But this isn't a debate over the pros and cons of goto. Goto is not relevant. This proposal is for dynamic scoping, not a way to change execution order or jump from one part of code to another.
This isn't a proposal to subroutines, it's a proposal for subroutines in which all lexically scoped variables are implicitly nonlocal.
Language fail :-) "This isn't a proposal for subroutines, it's a proposal for subroutines" :-) I've read the proposal differently, for dynamic scoping, not lexical scoping plus implicit nonlocal. It may be that you could get similar results from either, but they are fundamentally different. Python already has a kind of dynamic "scoping" (of sorts) which you probably use every day and never think twice about it. When an exception occurs, the scope of the exception handlers is the runtime call stack, not the lexical scope. There's nothing *inherently* bad about dynamic resolution of FOO, for any value of FOO (be it inheritence and method resolution, attribute lookup, scoping, or exception handling). It has its pros and cons, of course. I'd even allow that if overused or misused, dynamic scoping will be hard to reason about. (I believe it was Guy Steele who proved that dynamic scoping is equivalent to a single global scope, but of course that only applies when *all* functions are *always* dynamically scoped.) Just as we allow global variables for those odd cases where they are needed (or even just because it is more convienent), so there's a case for allowing dynamic scoping. Especially when the alternatives (classes, closures) are clumsy, difficult to get right, full of duplicated code and boilerplate. I wouldn't want every function call to use dynamic scoping, but there are times where I have really wanted to access the caller's environment, not the environment where my function was defined. We ought to give this concept a fair shake, not just dismiss it on the basis of platitudes "X considered harmful", especially when X isn't even relevant.
That's basically Goto
Its nothing like goto, except to the (small) degree that any jump to a subroutine is a goto.
Richard Stallman doesn't. I'm sure you can still find thousands of Python developers who regret that it uses significant whitespace and no braces. I know you can find Python developers who hate "lambda", and I know some who don't understand comprehensions and regret their inclusion in the language. Just about every proposal for new syntax is objected to, sometimes vehemently enough to drive Guido into "permanent vacation" (retirement) from his position of BDFL :-( So yes, I'm sure that some Emacs developers think dynamic scoping is a mistake, just as some Python developers think FOO is a mistake, for just about any value of FOO you can think of. -- Steve
On Thu, Aug 16, 2018 at 1:28 AM, Steven D'Aprano <steve@pearwood.info> wrote:
what am I missing? can't you get that by passing locals() in to a function? A bit clunky, sure, but everyone seems to agree that this a fairly rarely used approach. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
Hi Chris Steve and you wrote:
what am I missing? can't you get that by passing locals() in to a function?
I think this will fail when values are changed. According to https://docs.python.org/3/library/functions.html#locals
The contents of this dictionary should not be modified; changes may not affect the values of local and free variables used by the interpreter.
I suspect that Jacob, the original poster, wants to change state, and also to read changed state. Jacob, could you confirm or correct, that you want the called function to be able to change state in caller (or perhaps even further up the stack). To finish, here's an interactive example of changing the value of locals().
-- Jonathan
On Thu, Aug 16, 2018 at 10:33 AM, Jonathan Fine <jfine2358@gmail.com> wrote:
and: """ locals() Update and return a dictionary representing the current local symbol table. Free variables are returned by locals() when it is called in function blocks, but not in class blocks. """ it a dictionary *representing* the current local symbol table, so not the actual symbol table -- which surprised me, I thought that is was -- and sometimes it appears to be -- see later example:
To finish, here's an interactive example of changing the value of locals().
me too I would have thought "loc" would be in there.
now "loc is there"
hmm -- made me think that generators are doing something different here -- and indeed they are. If you use regular functions: In [30]: def local_modifying(loc): ...: """ ...: adds a "fred" key to the dict passed in ...: """ ...: print("locals passed in:", loc) ...: loc['fred'] = 5 ...: print("locals after adding", loc) ...: In [31]: def test_locals(): ...: """ ...: a simple local namespace to use ...: """ ...: a = 1 ...: b = 2 ...: local_modifying(locals()) ...: # does "fred" exist? ...: print(locals()) ...: # and we can access it the usual way ...: print("fred:", fred) ...: In [32]: test_locals() locals passed in: {'b': 2, 'a': 1} locals after adding {'b': 2, 'a': 1, 'fred': 5} {'b': 2, 'a': 1, 'fred': 5} fred: 5 It seems you CAN modify the locals dict passed in, and the change will show up in the enclosing scope. But it sounds like that is not guaranteed by the language. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
one more thought: given that:
I wonder why locals doesn't return a Mapping Proxy, or other read-only mapping object? If it's not guaranteed to be THE locals dict, and changes *may* not affect the real one (but may), a read-only object seems like much safer idea. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
чт, 16 авг. 2018 г. в 22:37, Chris Barker via Python-ideas < python-ideas@python.org>:
I have the same feeling and already have asked the same question before https://mail.python.org/pipermail/python-list/2018-February/731241.html. The main take away was that it is the CPython implementation detail and that `MappingProxy` type was introduced only in 3.3. I still do not find these arguments convincing. On the other hand with the current status quo - `locals` returns `dict` inside a function, we have only advantages: a) `locals` always returns a dict and is consistent in all places. (To be consistent is always good.) b) This type of question occurs every 6 months (there is always a topic to talk about at ML). c) When someone encounters this behavior first time, he thinks that he found a bug <- There was no error but nothing happened (gives an opportunity to study the internal details of the CPython implementation, the average level increases. Everyone also tries to find a way, something like "I believe, there must be a way, because Python is so dynamic" which theoretically allows you to make even more new discoveries.). d) This is considered by some as a way not to leak the CPython implementation details. (Here I have no comments, let it be on the conscience of others.) e) It leaves a room for a future changes (In fact, in some situations I would like to have this possibility). But if seriously, I think that the present situation is pretty weird especially when there is already the `MappingProxy` type. When it's not allowed and does not work (I'm only talking about the functions), it's better to raise early, instead of be silent until the last. With kind regards, -gdg
On Mon, Aug 20, 2018 at 12:43 AM, Kirill Balunov <kirillbalunov@gmail.com> wrote:
Related to (e) is that there is room for other implementations to permit changes to locals(), and furthermore, a fully-compliant Python implementation may use an actual dictionary for locals, and simply return that. (That CPython doesn't is an implementation detail for the sake of performance.) Requiring that it be a proxy would impose unnecessary cost on the implementation, without benefiting any compliant use. ChrisA
вс, 19 авг. 2018 г. в 17:51, Chris Angelico <rosuav@gmail.com>:
Let me disagree with you. While CPython is only one of the implementations of the Python language, it is the most common one and defacto is considered as a standard. Therefore, all the others, to be compliant, try to replicate all subtleties and features of the basic implementation - CPython. And even more so, I do not believe that anyone will rely or use such a feature. I'm not saying that this change should be a part of the Python language specification. I just said that this is already implied by the implementation of CPython3 and if you make it more clear with `MappingProxy` type it will be a net win in my eyes. With kind regards, -gdg
On Sun, Aug 19, 2018 at 06:18:56PM +0300, Kirill Balunov wrote: [...]
e) It leaves a room for a future changes (In fact, in some situations I would like to have this possibility).
[Chris]
I would like to see CPython locals() return a mapping object which raises a warning when you mutate it. * Currently, assignments to locals() simply do the wrong thing (modify the dict but fail to modify the local namespace) silently; such errors should never pass silently. * It should be a warning, not an error, since there are use-cases for modifying the local dict without caring about the namespace; in those cases we can simply silence the warning or ignore it. * Introducing a warning makes it clear that this is not a de facto language standard, but a mere implementation detail subject to change if somebody comes up with a better optimization for locals. - Other implementations should be permitted to use local namespaces which are read/write (that's a Quality Of Implementation issue). - Given that uses of locals() inside a function are already restricted and presumably uncommon, the extra cost of a warning is unlikely to be meaningful. [Kirill]
Not in this case. If CPython intended this to be the language behaviour, it should specify that limitation instead of merely giving a doc warning that changes "may not" affect the local variables. In fact, the warning in the docs is too strong. Modifying locals() inside a class or module scope is allowed, and works fine. It is only functions which is problematic.
Therefore, all the others, to be compliant, try to replicate all subtleties and features of the basic implementation - CPython.
Implementations are expected to obey the documented behaviour (unless given special dispensation to break the rules, which I think has happened once or twice with MicroPython). For behaviour which isn't dpcumented, it is true that implementations *often* try to copy CPython, but that's not mandatory and there are exceptions. PyPy is probably the alternate implementation which tries the hardest to duplicate CPython, but even PyPy has differences: http://doc.pypy.org/en/latest/cpython_differences.html including a few things which the PyPy developers believe is a bug in CPython. This list of differences between Jython and CPython is terribly out of date, but I haven't found anything more recent: http://www.jython.org/archive/21/docs/differences.html
And even more so, I do not believe that anyone will rely or use such a feature.
Users of CPython frequently rely on CPython implementation details. Why do you think Jython and IronPython users will be different? -- Steve
On Tue, Aug 21, 2018 at 12:21 AM, Steven D'Aprano <steve@pearwood.info> wrote:
That, I can get behind. Requiring it to be a proxy would IMO be a bad idea; but the warning here would be a CPython implementation detail. OTOH, warnings are easy to miss. But on the gripping hand, if more of Python and CPython raised warnings when odd things or implementation-specific things were done, it would encourage people to develop with warnings active. ChrisA
Hi I've created new thread === Documentation of locals() https://mail.python.org/pipermail/python-ideas/2018-August/052843.html Summary: There's prior art in bug.python.org relating to off-thread topic discussion of locals(). Suggest work on closing open documentation issues relating to locals(). [...] I suggest that those that have time, energy and interest focus on closing the open locals() documentation issues. Such documentation would, I think, provide an excellent starting point for any proposals to change behaviour. There are 13 open issues with 'locals' in the title. === Perhaps the discussion of locals() here could be moved to the new thread. -- Jonathan
On Mon, Aug 20, 2018 at 7:21 AM, Steven D'Aprano <steve@pearwood.info> wrote:
defacto standards are sub-optimum -- the docs say "may not" -- that seems really sketchy to me. Even if there is no change to the implementation of cPython, I'd like to see the behavior clearly defined -- if I pass a object returned by "locals()" to a function, and that function modifies that object -- will, or will not, the local namespace be altered? Saying it "may" be altered is kind of crazy! Does that mean the same code will have a different effect if run in two different (compliant) implementations of Python? That sure seems like a bad idea...
more so, I do not believe that anyone will rely or use such a feature.
well, this thread exists because someone wanted to do something like that -- i.e. manipulate the calling namespace from within a function. I suggested that passing locals() in might work -- it didn't work (in cPython), but if it HAD worked in whatever implementation the OP is using, then, yes, someone would now be relying on that feature. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Tue, Aug 21, 2018 at 2:16 AM, Chris Barker via Python-ideas <python-ideas@python.org> wrote:
If you do the wrong thing, a compliant Python is allowed to do one of two things. It's no different from anything else that's been defined as the wrong thing - opening a file and then dropping it on the floor (might close it promptly, might not), testing integers by identity ("x is 5"), etc, etc. The language is well-enough-defined that you can be confident that locals()["x"]=1 will EITHER change x to 1 OR have no significant effect. If you truly want to mandate that every Python implementation absolutely perfectly imitate CPython, what is the point of other implementations? Or a language spec? They'd just be refactorings of CPython. ChrisA
пн, 20 авг. 2018 г. в 17:23, Steven D'Aprano <steve@pearwood.info>:
I think I was pretty clear, that I only spoke about `locals` behavior inside functions and all its derivatives. At least I tried :) Talking about class and module, as for me, modifying `locals()` inside a class or module scope is allowed but is far from the right way to do. The phrase `may not affect` is the worst phrase for deterministic system. Your program may work or may not work correctly depending on the underlying Python implementation, but you would not know about this because there will be no error and no warning. Good luck fixing such bugs.
I think you are talking about recent discussion of MicroPython's `dict` implementation (that the order is not guaranteed?) It is a necessary trade off for them given their niche (if I understand correctly the explanation given by Paul Sokolovsky).
Hmm, what do you mean? It seems to me that everything happens the other way round. For example, recent attempts of numpy to make numpy more PyPy friendly. In fact, I do not know anyone who specifically rely on CPython implementation details in their production code. with kind regards, -gdg
On Thu, Aug 16, 2018 at 12:37 PM, Chris Angelico <rosuav@gmail.com> wrote:
I've no idea what interpreter you're using, but it doesn't work for me.
That was in iPython, with python3.6.2 -- I wouldn't have expected it to be different in this case though -- really odd. OK -- tired again, and it indeed it failed -- so I'm really confused. You can see my console output -- it did indeed work at least once.
You've changed a cached dictionary but haven't actually created a local
which still makes me wonder WHY locals() returns a writable dict... -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
By the way— really kludgy, doesn’t exec() do what you want here: Note The default *locals* act as described for function locals() <https://docs.python.org/3/library/functions.html#locals> below: modifications to the default *locals* dictionary should not be attempted. Pass an explicit *locals*dictionary if you need to see effects of the code on *locals* after function exec() <https://docs.python.org/3/library/functions.html#exec> returns. Though great still may not effect the current local namespace. But I’d be really surprised if there were no way to modify the local namespace — you can mess with pretty much anything else in Python. -CHB Sent from my iPhone On Aug 17, 2018, at 12:46 PM, Chris Barker <chris.barker@noaa.gov> wrote: On Thu, Aug 16, 2018 at 12:37 PM, Chris Angelico <rosuav@gmail.com> wrote:
I've no idea what interpreter you're using, but it doesn't work for me.
That was in iPython, with python3.6.2 -- I wouldn't have expected it to be different in this case though -- really odd. OK -- tired again, and it indeed it failed -- so I'm really confused. You can see my console output -- it did indeed work at least once.
You've changed a cached dictionary but haven't actually created a local
which still makes me wonder WHY locals() returns a writable dict... -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Thu, Aug 16, 2018 at 09:34:57AM -0700, Chris Barker wrote:
I want to pull in the caller's environment, not require the caller to push their environment in to me. Most of the time I just want read-access to the environment, but if I need to modify it, and the OP needs to do that, modifying locals() is unreliable. In general, it just silently fails, because locals() returns a copy of the local namespace. def spam(d): d['a'] = 'spam' def eggs(): a = 'ham' spam(locals()) print(a) Running that under CPython prints "ham", not "spam". (To be precise, writing to locals() works when you are in the module scope, or a class body, but not a function body. That's a CPython implementation detail; IronPython and Jython may behave differently.) -- Steve
On Wed, Aug 15, 2018 at 01:52:27PM -0500, Jacob Solinsky wrote:
I think that the standard term for what you want is *dynamic scoping*. http://wiki.c2.com/?DynamicScoping http://leafo.net/guides/dynamic-scoping-in-lua.html The most common language today that uses dynamic scoping is probably Emacs Lisp. Although dynamic scoping is very powerful, it is also harder to implement correctly (so I believe) and harder to reason about. It is generally considered that dynamic scoping is equivalent to the exclusive use of global variables (all functions see the same variables, no matter where they are defined), so the use of dynamic scoping gives up the benefits of local variables. (See also "Global variables considered harmful".)
This would generally be considered a severe violation of the Law of Demeter, one of the most important principles for reducing coupling between software components: https://www2.ccs.neu.edu/research/demeter/demeter-method/LawOfDemeter/LawOfD... http://www.ccs.neu.edu/home/lieber/LoD.html but also one of the most misunderstood. (Anyone who tells you that the LoD is "never use more than one dot" is wrong.) As far as I am concerned, reducing coupling between software components is the single most important design principle there is. But what do we do when the problem you are trying to solve inherently has a lot of coupling between data components? I think that computer science doesn't have a lot to say about such hard problems, apart from "avoid them".
What you quoted was not a description of the task, but a concrete implementation of one possible solution to the task. You are describing *how* you want to do it, but you haven't described what *it* is. This is not the place for it, but generally you should describe the functional requirements, not the implementation. You have Nouns, and Verbs, but what do you want to do with them? What is the high-level behaviour? If you focus on implementation too early, you may blind yourself to simpler solutions that accomplish the same end using a completely different means. A light-hearted example: "How do I turn the temperature of my refridgerator up? I need to boil some water, but the temperature control of the fridge only goes up to about 10°C (50°F)." "Why don't you use the stove or microwave oven?" "Because... that is to say... I... okay." Since I don't understand your functional requirements, I don't know whether you are dealing with a situation which inherently requires high coupling, or if you are missing a solution which would avoid the need for global variables or dynamic scoping. [...]
This suggests to me that a solution might be to generate and pass around some sort of *context object* for each word you operate on. The obvious problems with this include: - you have to generate that context object for every word, whether it will use it or not; - in principle, the amount of context needed could be unbounded; one word may only need information from the previous word, while another word in a long run-on sentence may need information from half a page back.
Closures are the exact equivalent of classes. If you have a solution (however clumsy) that works with classes, you should be able to re-write it in terms of closures. Whether that solution is less clumsy remains to be seen! So you can have something like this: def environment(): a = 1 b = 2 # define some closures def change1(): nonlocal a a += 1 def change2(): nonlocal a, b a += 2 b += 3 def inspect(): nonlocal a, b print(a, b) return (change1, change2, inspect) With those three closures, I can modify and inspect the variables defined by environment. Whether this technique simplifies your problem, I don't know. But if it is, it is surely easier than (1) convincing the core devs to add a new function call rule to Python, and (2) waiting for it to be implemented and released (which cannot be before version 3.8). (And yes, I agree that both the class-based and closure-based solution requires a lot of boilerplate, which is a bad thing.) -- Steve
tl;dr A pedantic note on dynamic binding. Steven D'Aprano writes:
The most common language today that uses dynamic scoping is probably Emacs Lisp.
AFAIK all Common-Lisp-ish languages implement dynamic scoping: anything defvar'd is dynamically-scoped.
Although dynamic scoping is very powerful, it is also harder to implement correctly (so I believe)
In fact, lexical scoping is harder to do correctly because you need to know what you're doing (determine whether a name is in the lexical scope). (Old) Emacs Lisp-style dynamic scoping (everything is dynamically scoped) is (conceptually) easy to implement correctly (but inefficiently!) simply by using a stack of bindings and doing all accesses by searching that stack top to bottom for a relevant binding. It's a pain in the butt to use *correctly* because every such binding can affect anything else in your program and any libraries you import. That's why Common Lisp and modern Emacs Lisp are lexically-scoped.[1] It's not that much harder to implement the dynamic scoping of defvars; you just have to add a check for the symbol being a defvar at binding time, and if not put it in the local namespace. It's true (as you write later in the thread) that Richard Stallman is still enamoured of dynamic scoping. The nicest thing I can say about Richard is that he has enormous capacity to hold huge complex structures in his head. The next nicest thing I can say about Richard is that he's an anomoly. If his enthusiasm for dynamic scoping is correct, it's a "stopped clock right twice a day" phenomenon. What's good for Richard may be *convenient* for the rest of us when hacking up one-offs, but I firmly believe that if *we* want to write reliable, nearly correct software for complex systems, dynamic scoping should be used sparingly at most. After 2 decades of maintaining Emacsen, I'd go so far as to say that Python's "global" and "nonlocal" declarations are distinctly preferable to Lisp's "defvar" and "special" declarations where Python's much more restricted constructs can do the job.
It is generally considered that dynamic scoping is equivalent to the exclusive use of global variables
That depends on your definition of "equivalent" and "global". At least psychologically, they can be very different regardless of the definition of "variable". In (old) Emacs Lisp, defun, lambda, and let all appear to define local variables, but they don't. They define local bindings of global variables, which affect any function called recursively from the function that does the binding that uses those variables as globals. And in any Lisp, it is in principle[2] impossible to write a correct program using dynamic scope if you import another module, because even if you've checked that module and there are no free occurrances of a given name at write-time, the module could be updated to use a free variable by that name before you run it and you will shadow its expected binding at run-time. To be safe, you need to implement lexical scope! In languages where "global" names are bound to memory locations at compile time, like C, or in Python where "global" means module scope, you don't have this problem. Sure, in C you can declare a variable "extern" or in Python you can import a name from another module, but that solves the problem: without an *explicit* declaration of that kind you know you can't affect another module. The point, of course, is not so much the "in principle" hazard, but rather the need to determine what the scope of a given binding *could* be and to check large extents of source code for bugs due to unexpected shadow bindings of free variables in order to be sure your program is correct, and that there is no restriction in principle on how large that extent might be. Steve Footnotes: [1] Technically, I think Emacs still defaults to dynamic scope and you have to turn lexical scope on with a pragma. [2] Again, technically, you can use dynamic scope safely in Common Lisp by interning your dynamic variables in a package, and so restricting their scope to your module plus any other modules that deliberate import those names.
If I understand well the need (I'm unsure I've had it myself), it would be easier to be able to import the function in the active context, like this: def foo(a): return a + c def bar(a, c): return foo(a) def bazz(a, c): import __file__.foo return foo(a) c = 5 call = bar(1, 3) # -> 6 jump = bazz(1, 3) # -> 4 bazz would be equivalent to: def bazz(a, c): def foo(a): return a + c return foo(a) I'm not saying the syntax is the one that should be used (__file__ possibly not existing may be a problem), not even saying that we should have this. I'm just showing a way to do the same thing with an easier (to me) syntax. - Brice
On Thu, Aug 16, 2018 at 10:31:28AM +0200, Brice Parent wrote:
The problem isn't that __file__ doesn't exist, it is that import DOES exist and does something completely unsuitable: def bazz(a, c): # Make the *name* foo a local foo = globals()['__file__'].foo # (roughly) return foo(a) But this doesn't change the scoping rules used when calling foo. It still runs using lexical scope, which means the a+c line in foo() still refers to the global variable c. What we would need is a completely different command, not "import", perhaps "load": def bazz(a, c): foo = load('foo') return foo(a) which would be equivalent to this pseudocode: def bazz(a, c): get the source code of foo (hopefully still available!) foo = compile(source code as inner function) call foo or possibly: def bazz(a, c): make a copy of foo and its bytecode change all(?) the LOAD_GLOBAL bytecode to LOAD_DEREF call the modified copy of foo Both implementations seem clunky, fragile and likely to be slow to me. -- Steve
Hi Jacob Thank you for your problem. I'll focus on your simple example (lightly edited for clarity)
This may allow you to solve your verb and morpheme problem. But I hope there's an easier way. One that can be done using Python as it is now. My understanding is that you want a stack. Similar to the Python's function call stack, but with something extra. So that functions execute in an richer context. Here's an idea. Start by giving us a toy example of a calculation you'd like to do. Then the problem is: how to do this in Python. By the way, are there any existing Python libraries for this sort of thing? I hope this helps. -- Jonathan
On Thu, Aug 16, 2018 at 4:52 AM, Jacob Solinsky <jacobsolinsky@gmail.com> wrote:
-Jumping to a function as opposed to calling a function
When a function is jumped to, it inherits the variables in the caller’s local namespace and is free to modify them or add new local variables, unlike a normal function call, wherein the caller’s namespace is inaccesible. At present, the only way I know of to accomplish this is to bundle all variables in the caller method’s namespace as properties of the method’s instance and have the callee method modify those properties. Though it is easier to read code written this way, it resulted in a great deal of redundancy in the code I was writing.
You're trying to shed encapsulation (by having the jumped-to function operate in the "caller's" namespace), but also pass parameters to it. That's going to cause a lot of confusion. I'm sympathetic to the problem, but I don't think functions are the solution here. You want some form of code block. There've been a number of proposals along those lines, and so far, not one of them has been really implementable. Check out some of the prior theories and see if one of them will work for you. ChrisA
Jumping into functions that mutate variables in the calling scope sounds a lot like "GoTo" <https://en.wikipedia.org/wiki/Considered_harmful> which is notorious for leading to code that's very hard to reason about. Your functions would implicitly require that you assign variables in the calling scope before calling the function instead of explicitly requiring them in the function's signature. That can be very confusing. A lot of times, when I find that my function call signatures have become unwieldy, it's a sign that I need to bundle a lot of related variables into an object and either define methods on that object or pass that object to functions instead of the large number of variables that it encapsulates. You just use 'self' as a sort-of back-pack that carries around what you need. It's difficult to see what you're after from your Verb and Morpheme example. They don't seem to call any functions that use variables from the caller's scope. Can you flesh that out a bit better? On Wed, Aug 15, 2018 at 3:24 PM, Chris Angelico <rosuav@gmail.com> wrote:
On Thu, Aug 16, 2018 at 7:40 AM, Abe Dillon <abedillon@gmail.com> wrote:
Drop the word "function" and it's no longer about a goto - it's about a block of code that can be deposited into another context. If you were to copy and paste that code, its meaning would be defined by where it is used; the idea of a "code block" is that you can do that with something that can be passed around like a function, but isn't a function. We could borrow a term from the Python grammar and call it a "suite", perhaps, but "code block" is probably the best term. As mentioned, there have been previous proposals along these lines. By the way:
It's difficult to see what you're after from your Verb and Morpheme example.
You're responding to MY post, which is a response to Jacob's. Please watch your use of pronouns; it would help a lot if you weren't top posting, and were directly responding to specific people's specific comments. ChrisA
[Chris Angelico]
Drop the word "function" and it's no longer about a goto - it's about a block of code that can be deposited into another context.
The only context I've used goto is in a TI-83 scientific calculator, but in that, it was just about jumping around to different labels in code and had nothing to do with functions. Though this may not have been the "canonical" goto that Dijkstra considered harmful. [Chris Angelico]
By the way:
It's difficult to see what you're after from your Verb and Morpheme example.
Sorry, I ran into a lot of problems trying to use the Google Groups interface to post, so I've resorted to strictly using email. Unfortunately, that makes the structure of the conversation less obvious to me. I'll try to be more mindful of this in the future. On Wed, Aug 15, 2018 at 4:58 PM, Chris Angelico <rosuav@gmail.com> wrote:
What I had hoped to do was use a preamble code block to collect all of the most common queries called by the mutate function in the local namespace, for example C = 'bpgkdtszSZjCmnywh' M = 'mn' class Morpheme: #stuff def preamble(self): ps = self.precedingmorpheme.form ss = self.succeedingmorpheme.form ssc = re.match(f'^[{C}]', self.succedingmorpheme.form) #Whether or not the following morpheme is consonant initial ssm = re.match(f'[{M}]$', self.precedingmorpheme.form) #Whether or not the preceding morpheme is nasal final these local variables are used quite often in the mutate methods, of which there are several dozen, so storing them by default saves a lot of typing. class Ban(Morpheme): def __init__(self): super().__init__(verb, precedingmorpheme, succeedingmorpheme, form = 'ban') def mutate(self): self.preamble()% if ps[-1] == 'd': self.form = 'p' + self.form[1:] self.preceding.form = '' if ssc: self.form = self.form + 'E' if ssm: self.form = re.sub('an', 'An', self.form) What I actually ended up doing is placing these queries in the Morpheme __init__ method def __init__(self, ...): self.ps = self.precedingmorpheme.form self.ss = self.succeedingmorpheme.form self.ssc = re.match(f'^[{C}]', self.succedingmorpheme.form) #Whether or not the following morpheme is consonant initial self.ssm = re.match(f'[{M}]$', self.precedingmorpheme.form) #Whether or not the preceding morpheme is nasal final This works perfectly fine, but I found cluttering the Morpheme object instances with flags and such to be inelegant, since these flags are only used by the mutate method. Also, without using a hacky solution like making Morpheme a subclass of types.SimpleNamespace, every new flag I might want to inject has to have a default value set in the __init__ method to prevent an AttributeError from being raised. Anyways, I will look around in the mail list for discussions of Code blocks, now that I know they are called. On Wed, 15 Aug 2018, 16:41 Abe Dillon, <abedillon@gmail.com> wrote:
On Thu, Aug 16, 2018 at 8:53 AM, Jacob Solinsky <jacobsolinsky@gmail.com> wrote:
Ahh, I see what you mean. I was thinking the other way - a single mutate function that has a part in there saying "now run the code block for this particular subclass". What would be really nice would be something like: from preamble() import * I'm fairly sure that (per se) isn't going to fly, but it would be interesting as a concept. Would need a new spelling. The quirky part of my brain is trying to figure if class namespace can be abused for this. ChrisA
[Jacob Solinsky]
these local variables are used quite often in the mutate methods, of which there are several dozen, so storing them by default saves a lot of typing.
There are several things you can do to alleviate the typing problem: 1) Get an IDE that has auto-complete. I recommend PyCharm Community Edition, but there are several. 2) Use better naming: there are many shorter synonyms for "preceding" and "succeding" and the word "morpheme" is redundant. 3) define as much as you can in the higher scopes: C = re.compile("^[bpgkdtszSZjCmnywh]").match M = re.compile("^[mn]$").match class Morpheme: @property def pf(self): return self.pre.form @property def sf(self): return self.suc.form @property def precm(self): return C(self.pf) @property def sucmm(self): return M(self.sf) ... I think what would help most of all is reorganizing your code. As I understand it, a morpheme is the "smallest grammatical unit in a language" <https://en.wikipedia.org/wiki/Morpheme> so maybe it makes more sense to make the morpheme class fairly simple and not have a morpheme have any inherent awareness of its preceding or succeeding morphemes. That seems like the job of a larger construct that would deal with sequences of morphemes. The way you have it right now, the morpheme seems to play a double role a linked-list and a gramatical unit. [Jacob Solinsky]
I found cluttering the Morpheme object instances with flags and such to be inelegant, since these flags are only used by the mutate method.
That may be an indicator that you're trying to do too much with the morpheme class. [Jacob Solinsky]
Yes, you typically want to define all your object attributes in __init__ otherwise your objects become fragile and can break if you don't call the right methods in the right order. It's generally considered a bad idea to define instance attributes anywhere else unless it's something like a "private" helper method that gets called inside __init__. [Jacob Solinsky]
Anyways, I will look around in the mail list for discussions of Code blocks, now that I know they are called.
Don't take this the wrong way, but it might be better to acquaint yourself with the set of tools already available before looking for new ones.
Jacob Solinsky writes:
What you're asking for is basically a "Ruby block", AFAICS. This has been requested many times, but most of the senior Pythonistas consider that it leads to a very high risk of unreadable and/or buggy code at no gain in power (there are several ways to efficiently accomplish what you want to do, you just don't like them[1]), and few cases where it's more readable/less buggy. I think if you search "ruby block site:mail.python.org" you'll probably bring up the threads. You won't find a lot of encouraging discussion, I'm afraid. What I would do in your case is class Morpheme: class Preamble: def __init__(self, m): # "m" for "morpheme" self.ps = m.precedingmorpheme.form self.ss = m.succeedingmorpheme.form self.ssc = re.match(f'^[{C}]', m.succedingmorpheme.form) self.ssm = re.match(f'[{M}]$', m.precedingmorpheme.form) def mutate(): p = self.Preamble(self) # use p.ps, p.ss, etc It's a little less efficient, of course, because of overhead managing the Preamble objects. Another possibility would be to use properties, so that instead of *storing* these temporary variables on the Morpheme, they would be computed on the fly: class Morpheme: @property def ps(self): return self.precedingmorpheme.form @property def ss(self): return self.succeedingmorpheme.form @property def ssc(self): return re.match(f'^[{C}]', self.succeedingmorpheme.form) @property def ssm(self): return re.match(f'[{M}]$', self.precedingmorpheme.form) and so on. Then you can access them with "self.ps" etc in the mutate method. This might be considered elegant (although verbose in the Morpheme definition) if these properties are usually used only once per mutate call, and "expensive" properties like ssc and ssm are never called more than once. (If they are called multiple times, it's easy to create a cache.) Of course property accesses do add method call overhead, but they look nicer than method call syntax. On the other hand, if "many" properties are expensive, and "most" mutate methods use only a few of these, you could save runtime this way, since the expensive methods are called only as-needed. I don't understand the "need default value in __init__" issue, since "inject" implied to me that you initialize before use. But if that's a problem for ordinary attributes, I presume it'll be more annoying with properties because you have to write a method for it. So I don't know if this is helpful at all. Footnotes: [1] "I don't like it" is a sufficient reason to propose a change. I'm simply saying you have the *power* to do what you want.
Sorry for the double post, but I wanted to make sure you saw my original misplaced post: Jumping into functions that mutate variables in the calling scope sounds a lot like "GoTo" <https://en.wikipedia.org/wiki/Considered_harmful> which is notorious for leading to code that's very hard to reason about. Your functions would implicitly require that you assign variables in the calling scope before calling the function instead of explicitly requiring them in the function's signature. That can be very confusing. A lot of times, when I find that my function call signatures have become unwieldy, it's a sign that I need to bundle a lot of related variables into an object and either define methods on that object or pass that object to functions instead of the large number of variables that it encapsulates. You just use 'self' as a sort-of back-pack that carries around what you need. It's difficult to see what you're after from your Verb and Morpheme example. They don't seem to call any functions that use variables from the caller's scope. Can you flesh that out a bit more to show where the problem arrises?
Goto considered harmful. https://homepages.cwi.nl/~storm/teaching/reader/Dijkstra68.pdf&ved=2ahUKEwi8lqHYqfDcAhXImOAKHeAzDmsQFjAEegQIBhAB&usg=AOvVaw1CZug_36-PbevItYXTb7SR On Wed, Aug 15, 2018, 3:52 PM Jacob Solinsky <jacobsolinsky@gmail.com> wrote:
Hmm.. link broke. Is this right? https://www.google.com/url?sa=t&source=web&rct=j&url=https://homepages.cwi.nl/~storm/teaching/reader/Dijkstra68.pdf&ved=2ahUKEwi8lqHYqfDcAhXImOAKHeAzDmsQFjAEegQIBhAB&usg=AOvVaw1CZug_36-PbevItYXTb7SR On Wed, Aug 15, 2018, 8:35 PM David Mertz <mertz@gnosis.cx> wrote:
How about just https://homepages.cwi.nl/~storm/teaching/reader/Dijkstra68.pdf On Wed, Aug 15, 2018 at 5:38 PM David Mertz <mertz@gnosis.cx> wrote:
I have no opinion on the proposal, but the equation with goto is dead wrong. Yes, technically this is a goto. But so is a regular function call. Jumping to a function makes thd code _easier_ to reason about, statically, not harder. When you call an arbitrary function, you have no idea whether it will return or not. When you jump to a function, you _know_ it will not return. That's strictly more information, based solely on the syntax of the statement. Yes, if you have local functions mutating the state then jumpung to them makes the code difficult to read. *but so is calling them*, with the addition of the returning that you have to track. Jumping to functions is procedural programming. There is no reasonfor it to be "considered harmful" any more than calling funcions; indeed even less so. Elazar On Wed, Aug 15, 2018, 17:42 Eric Fahlgren <ericfahlgren@gmail.com> wrote:
On Wed, Aug 15, 2018 at 08:35:35PM -0400, David Mertz wrote:
Goto considered harmful.
Fortunately this proposal has nothing to do with goto. Elazar is correct that its a kind of subroutine call, just like an ordinary function call, except the scoping rules are different. And for the record, not everyone agrees that Dijkstra is correct about goto. Certainly unstructured code is harmful, but we use restricted forms of goto all the time, we just don't call it by that name: - loops - continue - break - if...else - function calls - exception handling Just like goto, these are all jumps which change the execution order of your code. And some people defend limited, careful use of explicit goto, including Donald Knuth. -- Steve
When did you last read the Dijkstra paper. It's short and he explains very well why exactly all the constructs you mention are unlike Goto. This isn't a proposal to subroutines, it's a proposal for subroutines in which all lexically scoped variables are implicitly nonlocal. That's basically Goto, certainly course enough to suffer all of Dijkstra's complaints. Yes, technically this could be called dynamic scope instead. But notwithstanding the use in elisp, that's pretty widely regarded as a bad ideas (even by the emacs developers who regret that early decision). On Wed, Aug 15, 2018, 9:41 PM Steven D'Aprano <steve@pearwood.info> wrote:
On Wed, Aug 15, 2018 at 09:58:28PM -0400, David Mertz wrote:
When did you last read the Dijkstra paper. It's short and he explains very well why exactly all the constructs you mention are unlike Goto.
Of course I've read it, and no he does not. He only mentions if...else and looping. He doesn't mention break, continue, exceptions (I don't think exceptions had even been invented in 1968). I think that if Dijkstra were still alive, he would probably chose to understate or dismiss the similarities between goto and break/continue. Others might choose to focus on the similarities rather than the differences, e.g. "continue" is just a goto to the start of the loop, "break" is goto the end of the loop. But this isn't a debate over the pros and cons of goto. Goto is not relevant. This proposal is for dynamic scoping, not a way to change execution order or jump from one part of code to another.
This isn't a proposal to subroutines, it's a proposal for subroutines in which all lexically scoped variables are implicitly nonlocal.
Language fail :-) "This isn't a proposal for subroutines, it's a proposal for subroutines" :-) I've read the proposal differently, for dynamic scoping, not lexical scoping plus implicit nonlocal. It may be that you could get similar results from either, but they are fundamentally different. Python already has a kind of dynamic "scoping" (of sorts) which you probably use every day and never think twice about it. When an exception occurs, the scope of the exception handlers is the runtime call stack, not the lexical scope. There's nothing *inherently* bad about dynamic resolution of FOO, for any value of FOO (be it inheritence and method resolution, attribute lookup, scoping, or exception handling). It has its pros and cons, of course. I'd even allow that if overused or misused, dynamic scoping will be hard to reason about. (I believe it was Guy Steele who proved that dynamic scoping is equivalent to a single global scope, but of course that only applies when *all* functions are *always* dynamically scoped.) Just as we allow global variables for those odd cases where they are needed (or even just because it is more convienent), so there's a case for allowing dynamic scoping. Especially when the alternatives (classes, closures) are clumsy, difficult to get right, full of duplicated code and boilerplate. I wouldn't want every function call to use dynamic scoping, but there are times where I have really wanted to access the caller's environment, not the environment where my function was defined. We ought to give this concept a fair shake, not just dismiss it on the basis of platitudes "X considered harmful", especially when X isn't even relevant.
That's basically Goto
Its nothing like goto, except to the (small) degree that any jump to a subroutine is a goto.
Richard Stallman doesn't. I'm sure you can still find thousands of Python developers who regret that it uses significant whitespace and no braces. I know you can find Python developers who hate "lambda", and I know some who don't understand comprehensions and regret their inclusion in the language. Just about every proposal for new syntax is objected to, sometimes vehemently enough to drive Guido into "permanent vacation" (retirement) from his position of BDFL :-( So yes, I'm sure that some Emacs developers think dynamic scoping is a mistake, just as some Python developers think FOO is a mistake, for just about any value of FOO you can think of. -- Steve
On Thu, Aug 16, 2018 at 1:28 AM, Steven D'Aprano <steve@pearwood.info> wrote:
what am I missing? can't you get that by passing locals() in to a function? A bit clunky, sure, but everyone seems to agree that this a fairly rarely used approach. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
Hi Chris Steve and you wrote:
what am I missing? can't you get that by passing locals() in to a function?
I think this will fail when values are changed. According to https://docs.python.org/3/library/functions.html#locals
The contents of this dictionary should not be modified; changes may not affect the values of local and free variables used by the interpreter.
I suspect that Jacob, the original poster, wants to change state, and also to read changed state. Jacob, could you confirm or correct, that you want the called function to be able to change state in caller (or perhaps even further up the stack). To finish, here's an interactive example of changing the value of locals().
-- Jonathan
On Thu, Aug 16, 2018 at 10:33 AM, Jonathan Fine <jfine2358@gmail.com> wrote:
and: """ locals() Update and return a dictionary representing the current local symbol table. Free variables are returned by locals() when it is called in function blocks, but not in class blocks. """ it a dictionary *representing* the current local symbol table, so not the actual symbol table -- which surprised me, I thought that is was -- and sometimes it appears to be -- see later example:
To finish, here's an interactive example of changing the value of locals().
me too I would have thought "loc" would be in there.
now "loc is there"
hmm -- made me think that generators are doing something different here -- and indeed they are. If you use regular functions: In [30]: def local_modifying(loc): ...: """ ...: adds a "fred" key to the dict passed in ...: """ ...: print("locals passed in:", loc) ...: loc['fred'] = 5 ...: print("locals after adding", loc) ...: In [31]: def test_locals(): ...: """ ...: a simple local namespace to use ...: """ ...: a = 1 ...: b = 2 ...: local_modifying(locals()) ...: # does "fred" exist? ...: print(locals()) ...: # and we can access it the usual way ...: print("fred:", fred) ...: In [32]: test_locals() locals passed in: {'b': 2, 'a': 1} locals after adding {'b': 2, 'a': 1, 'fred': 5} {'b': 2, 'a': 1, 'fred': 5} fred: 5 It seems you CAN modify the locals dict passed in, and the change will show up in the enclosing scope. But it sounds like that is not guaranteed by the language. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
one more thought: given that:
I wonder why locals doesn't return a Mapping Proxy, or other read-only mapping object? If it's not guaranteed to be THE locals dict, and changes *may* not affect the real one (but may), a read-only object seems like much safer idea. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
чт, 16 авг. 2018 г. в 22:37, Chris Barker via Python-ideas < python-ideas@python.org>:
I have the same feeling and already have asked the same question before https://mail.python.org/pipermail/python-list/2018-February/731241.html. The main take away was that it is the CPython implementation detail and that `MappingProxy` type was introduced only in 3.3. I still do not find these arguments convincing. On the other hand with the current status quo - `locals` returns `dict` inside a function, we have only advantages: a) `locals` always returns a dict and is consistent in all places. (To be consistent is always good.) b) This type of question occurs every 6 months (there is always a topic to talk about at ML). c) When someone encounters this behavior first time, he thinks that he found a bug <- There was no error but nothing happened (gives an opportunity to study the internal details of the CPython implementation, the average level increases. Everyone also tries to find a way, something like "I believe, there must be a way, because Python is so dynamic" which theoretically allows you to make even more new discoveries.). d) This is considered by some as a way not to leak the CPython implementation details. (Here I have no comments, let it be on the conscience of others.) e) It leaves a room for a future changes (In fact, in some situations I would like to have this possibility). But if seriously, I think that the present situation is pretty weird especially when there is already the `MappingProxy` type. When it's not allowed and does not work (I'm only talking about the functions), it's better to raise early, instead of be silent until the last. With kind regards, -gdg
On Mon, Aug 20, 2018 at 12:43 AM, Kirill Balunov <kirillbalunov@gmail.com> wrote:
Related to (e) is that there is room for other implementations to permit changes to locals(), and furthermore, a fully-compliant Python implementation may use an actual dictionary for locals, and simply return that. (That CPython doesn't is an implementation detail for the sake of performance.) Requiring that it be a proxy would impose unnecessary cost on the implementation, without benefiting any compliant use. ChrisA
вс, 19 авг. 2018 г. в 17:51, Chris Angelico <rosuav@gmail.com>:
Let me disagree with you. While CPython is only one of the implementations of the Python language, it is the most common one and defacto is considered as a standard. Therefore, all the others, to be compliant, try to replicate all subtleties and features of the basic implementation - CPython. And even more so, I do not believe that anyone will rely or use such a feature. I'm not saying that this change should be a part of the Python language specification. I just said that this is already implied by the implementation of CPython3 and if you make it more clear with `MappingProxy` type it will be a net win in my eyes. With kind regards, -gdg
On Sun, Aug 19, 2018 at 06:18:56PM +0300, Kirill Balunov wrote: [...]
e) It leaves a room for a future changes (In fact, in some situations I would like to have this possibility).
[Chris]
I would like to see CPython locals() return a mapping object which raises a warning when you mutate it. * Currently, assignments to locals() simply do the wrong thing (modify the dict but fail to modify the local namespace) silently; such errors should never pass silently. * It should be a warning, not an error, since there are use-cases for modifying the local dict without caring about the namespace; in those cases we can simply silence the warning or ignore it. * Introducing a warning makes it clear that this is not a de facto language standard, but a mere implementation detail subject to change if somebody comes up with a better optimization for locals. - Other implementations should be permitted to use local namespaces which are read/write (that's a Quality Of Implementation issue). - Given that uses of locals() inside a function are already restricted and presumably uncommon, the extra cost of a warning is unlikely to be meaningful. [Kirill]
Not in this case. If CPython intended this to be the language behaviour, it should specify that limitation instead of merely giving a doc warning that changes "may not" affect the local variables. In fact, the warning in the docs is too strong. Modifying locals() inside a class or module scope is allowed, and works fine. It is only functions which is problematic.
Therefore, all the others, to be compliant, try to replicate all subtleties and features of the basic implementation - CPython.
Implementations are expected to obey the documented behaviour (unless given special dispensation to break the rules, which I think has happened once or twice with MicroPython). For behaviour which isn't dpcumented, it is true that implementations *often* try to copy CPython, but that's not mandatory and there are exceptions. PyPy is probably the alternate implementation which tries the hardest to duplicate CPython, but even PyPy has differences: http://doc.pypy.org/en/latest/cpython_differences.html including a few things which the PyPy developers believe is a bug in CPython. This list of differences between Jython and CPython is terribly out of date, but I haven't found anything more recent: http://www.jython.org/archive/21/docs/differences.html
And even more so, I do not believe that anyone will rely or use such a feature.
Users of CPython frequently rely on CPython implementation details. Why do you think Jython and IronPython users will be different? -- Steve
On Tue, Aug 21, 2018 at 12:21 AM, Steven D'Aprano <steve@pearwood.info> wrote:
That, I can get behind. Requiring it to be a proxy would IMO be a bad idea; but the warning here would be a CPython implementation detail. OTOH, warnings are easy to miss. But on the gripping hand, if more of Python and CPython raised warnings when odd things or implementation-specific things were done, it would encourage people to develop with warnings active. ChrisA
Hi I've created new thread === Documentation of locals() https://mail.python.org/pipermail/python-ideas/2018-August/052843.html Summary: There's prior art in bug.python.org relating to off-thread topic discussion of locals(). Suggest work on closing open documentation issues relating to locals(). [...] I suggest that those that have time, energy and interest focus on closing the open locals() documentation issues. Such documentation would, I think, provide an excellent starting point for any proposals to change behaviour. There are 13 open issues with 'locals' in the title. === Perhaps the discussion of locals() here could be moved to the new thread. -- Jonathan
On Mon, Aug 20, 2018 at 7:21 AM, Steven D'Aprano <steve@pearwood.info> wrote:
defacto standards are sub-optimum -- the docs say "may not" -- that seems really sketchy to me. Even if there is no change to the implementation of cPython, I'd like to see the behavior clearly defined -- if I pass a object returned by "locals()" to a function, and that function modifies that object -- will, or will not, the local namespace be altered? Saying it "may" be altered is kind of crazy! Does that mean the same code will have a different effect if run in two different (compliant) implementations of Python? That sure seems like a bad idea...
more so, I do not believe that anyone will rely or use such a feature.
well, this thread exists because someone wanted to do something like that -- i.e. manipulate the calling namespace from within a function. I suggested that passing locals() in might work -- it didn't work (in cPython), but if it HAD worked in whatever implementation the OP is using, then, yes, someone would now be relying on that feature. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Tue, Aug 21, 2018 at 2:16 AM, Chris Barker via Python-ideas <python-ideas@python.org> wrote:
If you do the wrong thing, a compliant Python is allowed to do one of two things. It's no different from anything else that's been defined as the wrong thing - opening a file and then dropping it on the floor (might close it promptly, might not), testing integers by identity ("x is 5"), etc, etc. The language is well-enough-defined that you can be confident that locals()["x"]=1 will EITHER change x to 1 OR have no significant effect. If you truly want to mandate that every Python implementation absolutely perfectly imitate CPython, what is the point of other implementations? Or a language spec? They'd just be refactorings of CPython. ChrisA
пн, 20 авг. 2018 г. в 17:23, Steven D'Aprano <steve@pearwood.info>:
I think I was pretty clear, that I only spoke about `locals` behavior inside functions and all its derivatives. At least I tried :) Talking about class and module, as for me, modifying `locals()` inside a class or module scope is allowed but is far from the right way to do. The phrase `may not affect` is the worst phrase for deterministic system. Your program may work or may not work correctly depending on the underlying Python implementation, but you would not know about this because there will be no error and no warning. Good luck fixing such bugs.
I think you are talking about recent discussion of MicroPython's `dict` implementation (that the order is not guaranteed?) It is a necessary trade off for them given their niche (if I understand correctly the explanation given by Paul Sokolovsky).
Hmm, what do you mean? It seems to me that everything happens the other way round. For example, recent attempts of numpy to make numpy more PyPy friendly. In fact, I do not know anyone who specifically rely on CPython implementation details in their production code. with kind regards, -gdg
On Thu, Aug 16, 2018 at 12:37 PM, Chris Angelico <rosuav@gmail.com> wrote:
I've no idea what interpreter you're using, but it doesn't work for me.
That was in iPython, with python3.6.2 -- I wouldn't have expected it to be different in this case though -- really odd. OK -- tired again, and it indeed it failed -- so I'm really confused. You can see my console output -- it did indeed work at least once.
You've changed a cached dictionary but haven't actually created a local
which still makes me wonder WHY locals() returns a writable dict... -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
By the way— really kludgy, doesn’t exec() do what you want here: Note The default *locals* act as described for function locals() <https://docs.python.org/3/library/functions.html#locals> below: modifications to the default *locals* dictionary should not be attempted. Pass an explicit *locals*dictionary if you need to see effects of the code on *locals* after function exec() <https://docs.python.org/3/library/functions.html#exec> returns. Though great still may not effect the current local namespace. But I’d be really surprised if there were no way to modify the local namespace — you can mess with pretty much anything else in Python. -CHB Sent from my iPhone On Aug 17, 2018, at 12:46 PM, Chris Barker <chris.barker@noaa.gov> wrote: On Thu, Aug 16, 2018 at 12:37 PM, Chris Angelico <rosuav@gmail.com> wrote:
I've no idea what interpreter you're using, but it doesn't work for me.
That was in iPython, with python3.6.2 -- I wouldn't have expected it to be different in this case though -- really odd. OK -- tired again, and it indeed it failed -- so I'm really confused. You can see my console output -- it did indeed work at least once.
You've changed a cached dictionary but haven't actually created a local
which still makes me wonder WHY locals() returns a writable dict... -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Thu, Aug 16, 2018 at 09:34:57AM -0700, Chris Barker wrote:
I want to pull in the caller's environment, not require the caller to push their environment in to me. Most of the time I just want read-access to the environment, but if I need to modify it, and the OP needs to do that, modifying locals() is unreliable. In general, it just silently fails, because locals() returns a copy of the local namespace. def spam(d): d['a'] = 'spam' def eggs(): a = 'ham' spam(locals()) print(a) Running that under CPython prints "ham", not "spam". (To be precise, writing to locals() works when you are in the module scope, or a class body, but not a function body. That's a CPython implementation detail; IronPython and Jython may behave differently.) -- Steve
On Wed, Aug 15, 2018 at 01:52:27PM -0500, Jacob Solinsky wrote:
I think that the standard term for what you want is *dynamic scoping*. http://wiki.c2.com/?DynamicScoping http://leafo.net/guides/dynamic-scoping-in-lua.html The most common language today that uses dynamic scoping is probably Emacs Lisp. Although dynamic scoping is very powerful, it is also harder to implement correctly (so I believe) and harder to reason about. It is generally considered that dynamic scoping is equivalent to the exclusive use of global variables (all functions see the same variables, no matter where they are defined), so the use of dynamic scoping gives up the benefits of local variables. (See also "Global variables considered harmful".)
This would generally be considered a severe violation of the Law of Demeter, one of the most important principles for reducing coupling between software components: https://www2.ccs.neu.edu/research/demeter/demeter-method/LawOfDemeter/LawOfD... http://www.ccs.neu.edu/home/lieber/LoD.html but also one of the most misunderstood. (Anyone who tells you that the LoD is "never use more than one dot" is wrong.) As far as I am concerned, reducing coupling between software components is the single most important design principle there is. But what do we do when the problem you are trying to solve inherently has a lot of coupling between data components? I think that computer science doesn't have a lot to say about such hard problems, apart from "avoid them".
What you quoted was not a description of the task, but a concrete implementation of one possible solution to the task. You are describing *how* you want to do it, but you haven't described what *it* is. This is not the place for it, but generally you should describe the functional requirements, not the implementation. You have Nouns, and Verbs, but what do you want to do with them? What is the high-level behaviour? If you focus on implementation too early, you may blind yourself to simpler solutions that accomplish the same end using a completely different means. A light-hearted example: "How do I turn the temperature of my refridgerator up? I need to boil some water, but the temperature control of the fridge only goes up to about 10°C (50°F)." "Why don't you use the stove or microwave oven?" "Because... that is to say... I... okay." Since I don't understand your functional requirements, I don't know whether you are dealing with a situation which inherently requires high coupling, or if you are missing a solution which would avoid the need for global variables or dynamic scoping. [...]
This suggests to me that a solution might be to generate and pass around some sort of *context object* for each word you operate on. The obvious problems with this include: - you have to generate that context object for every word, whether it will use it or not; - in principle, the amount of context needed could be unbounded; one word may only need information from the previous word, while another word in a long run-on sentence may need information from half a page back.
Closures are the exact equivalent of classes. If you have a solution (however clumsy) that works with classes, you should be able to re-write it in terms of closures. Whether that solution is less clumsy remains to be seen! So you can have something like this: def environment(): a = 1 b = 2 # define some closures def change1(): nonlocal a a += 1 def change2(): nonlocal a, b a += 2 b += 3 def inspect(): nonlocal a, b print(a, b) return (change1, change2, inspect) With those three closures, I can modify and inspect the variables defined by environment. Whether this technique simplifies your problem, I don't know. But if it is, it is surely easier than (1) convincing the core devs to add a new function call rule to Python, and (2) waiting for it to be implemented and released (which cannot be before version 3.8). (And yes, I agree that both the class-based and closure-based solution requires a lot of boilerplate, which is a bad thing.) -- Steve
tl;dr A pedantic note on dynamic binding. Steven D'Aprano writes:
The most common language today that uses dynamic scoping is probably Emacs Lisp.
AFAIK all Common-Lisp-ish languages implement dynamic scoping: anything defvar'd is dynamically-scoped.
Although dynamic scoping is very powerful, it is also harder to implement correctly (so I believe)
In fact, lexical scoping is harder to do correctly because you need to know what you're doing (determine whether a name is in the lexical scope). (Old) Emacs Lisp-style dynamic scoping (everything is dynamically scoped) is (conceptually) easy to implement correctly (but inefficiently!) simply by using a stack of bindings and doing all accesses by searching that stack top to bottom for a relevant binding. It's a pain in the butt to use *correctly* because every such binding can affect anything else in your program and any libraries you import. That's why Common Lisp and modern Emacs Lisp are lexically-scoped.[1] It's not that much harder to implement the dynamic scoping of defvars; you just have to add a check for the symbol being a defvar at binding time, and if not put it in the local namespace. It's true (as you write later in the thread) that Richard Stallman is still enamoured of dynamic scoping. The nicest thing I can say about Richard is that he has enormous capacity to hold huge complex structures in his head. The next nicest thing I can say about Richard is that he's an anomoly. If his enthusiasm for dynamic scoping is correct, it's a "stopped clock right twice a day" phenomenon. What's good for Richard may be *convenient* for the rest of us when hacking up one-offs, but I firmly believe that if *we* want to write reliable, nearly correct software for complex systems, dynamic scoping should be used sparingly at most. After 2 decades of maintaining Emacsen, I'd go so far as to say that Python's "global" and "nonlocal" declarations are distinctly preferable to Lisp's "defvar" and "special" declarations where Python's much more restricted constructs can do the job.
It is generally considered that dynamic scoping is equivalent to the exclusive use of global variables
That depends on your definition of "equivalent" and "global". At least psychologically, they can be very different regardless of the definition of "variable". In (old) Emacs Lisp, defun, lambda, and let all appear to define local variables, but they don't. They define local bindings of global variables, which affect any function called recursively from the function that does the binding that uses those variables as globals. And in any Lisp, it is in principle[2] impossible to write a correct program using dynamic scope if you import another module, because even if you've checked that module and there are no free occurrances of a given name at write-time, the module could be updated to use a free variable by that name before you run it and you will shadow its expected binding at run-time. To be safe, you need to implement lexical scope! In languages where "global" names are bound to memory locations at compile time, like C, or in Python where "global" means module scope, you don't have this problem. Sure, in C you can declare a variable "extern" or in Python you can import a name from another module, but that solves the problem: without an *explicit* declaration of that kind you know you can't affect another module. The point, of course, is not so much the "in principle" hazard, but rather the need to determine what the scope of a given binding *could* be and to check large extents of source code for bugs due to unexpected shadow bindings of free variables in order to be sure your program is correct, and that there is no restriction in principle on how large that extent might be. Steve Footnotes: [1] Technically, I think Emacs still defaults to dynamic scope and you have to turn lexical scope on with a pragma. [2] Again, technically, you can use dynamic scope safely in Common Lisp by interning your dynamic variables in a package, and so restricting their scope to your module plus any other modules that deliberate import those names.
If I understand well the need (I'm unsure I've had it myself), it would be easier to be able to import the function in the active context, like this: def foo(a): return a + c def bar(a, c): return foo(a) def bazz(a, c): import __file__.foo return foo(a) c = 5 call = bar(1, 3) # -> 6 jump = bazz(1, 3) # -> 4 bazz would be equivalent to: def bazz(a, c): def foo(a): return a + c return foo(a) I'm not saying the syntax is the one that should be used (__file__ possibly not existing may be a problem), not even saying that we should have this. I'm just showing a way to do the same thing with an easier (to me) syntax. - Brice
On Thu, Aug 16, 2018 at 10:31:28AM +0200, Brice Parent wrote:
The problem isn't that __file__ doesn't exist, it is that import DOES exist and does something completely unsuitable: def bazz(a, c): # Make the *name* foo a local foo = globals()['__file__'].foo # (roughly) return foo(a) But this doesn't change the scoping rules used when calling foo. It still runs using lexical scope, which means the a+c line in foo() still refers to the global variable c. What we would need is a completely different command, not "import", perhaps "load": def bazz(a, c): foo = load('foo') return foo(a) which would be equivalent to this pseudocode: def bazz(a, c): get the source code of foo (hopefully still available!) foo = compile(source code as inner function) call foo or possibly: def bazz(a, c): make a copy of foo and its bytecode change all(?) the LOAD_GLOBAL bytecode to LOAD_DEREF call the modified copy of foo Both implementations seem clunky, fragile and likely to be slow to me. -- Steve
participants (14)
-
Abe Dillon -
Brice Parent -
Chris Angelico -
Chris Barker -
Chris Barker - NOAA Federal
-
David Mertz -
Elazar -
Eric Fahlgren -
Jacob Solinsky -
Jonathan Fine -
Kirill Balunov -
Rhodri James -
Stephen J. Turnbull -
Steven D'Aprano