:Hello all,
I am sorry to send this out to the list and then be so silent, work and children and all. I am going to try and address the issues raised in this thread, with apologies if I miss someones issue. Some of these are addressed in the documentation I wrote, but I know I can't expect people to take their time to go off and read a lot of material, I am thankful enough for the time that people put even into reading and responding. This message might get long, again, my apologies. I have put names by the response to that person, so you can read your section. The response may refer to other responses, but this was long enough without duplication. Again I might not have been completely through with each response in light of the length, but tried to hit the high points and conceptual discussions, if not the details.
Ben Rudiak-Gould:
Your point with iteration is well taken. Depending on the variable bound, it may end up transparent to you without you needing to care, as in the case of something like my HistoricVar demo. However it is a fair point, that it might not be transparent and you could end up with an exception thrown at some point. I would argue this is not much different than using any library code where you would need to understand the api, but documentation is not always up to par, and it is indeed one more thing to think about or check for.
Your idea of marking a variable with something like metavar is an interesting idea, and definitely something I am going to think more on. If this is required for the new behavior in each function where it is used I don't think that is compatible with how I was envisioning this to work, it would require more thought. At this point one of the benefits of this type of metavar, is that I can declare it with whatever side effects I want (such as a callback on assignment) and pass it into existing third party code and not need that code to be altered, that code would not care that it is special.
This is a case where the sense of responsibility is flipped from what you are worried about though, where I would be responsible for creating an appropriate variable (like I am for types now) when I am using your library. I am not sure how that would play well with a keyword at declaration, except that someone could read my code (or a library using this type of variable) and see that it was defined in a special way.
Rhodri James:
Your point seems similar Ben's above so perhaps that addresses some of your points. I think the idea is in what context will this code be used. If I am a library author then I might not want to expose this type of variable to outside the api without some sort of annotation. That does not mean it could not be really useful inside an api, or for instance used as an instance level property.
Anders Hovmoller:
Apologies about not getting the accents in your name as I type this. I appreciate the devils advocate stance. If this proposal went anywhere or not, I was hopping to learn a lot through the discussion, and process. Continuing the discussion really helps to that end.
Chris Angelico:
In my proposal you are correct that in functions such as:
def f1():
x = metavar()
return x
def f2():
return metavar()
would not have worked as expected. As soon that the variable became bound to the name, the LOAD operations that happened before a RETURN operation would call __getself__ and the result would be returned. A built in method wold have been required to get to the underlying object, such as in
def f3():
x = metavar()
return getcloaked('x')
This means that those two functions would have resulted in different returns. (A similar problem would exist for function calls). I was never entirely happy with this behavior either.
A little while ago I pushed a patch up to my cpython fork that made the behavior the same as existing python, if you use a metavar in the return statement, it will be returned from the function to the calling scope, where it will again behave as a metavar (I also made the same change for calling a function with a metavar as an argument)
I hope some of the examples I linked to in previous messages helped, but I am happy to talk more about what the call chain looks like, and how I intend things to work.
Greg Ewing:
I think some of your points were also addressed above. I am happy to expand on any of them myself if the above has not addressed them.
Andrew Barnert:
I thank you for also playing devils advocate, and if not providing a "vote of confidence" then a willingness to make sure my ideas are heard, and that I have a chance to address and possibly make a better proposal.
Brendan Barnwell:
I can appreciate your position, and might well feel the same if I was a maintainer of the python code base. However, I will confess I did come away a little stung by the tone of your message. This may just be because it is a bit dearer to me, as it is something I have been laboring on.
I tried to write up documentation and do do diligence to have material ready before coming to the list as to not present a half baked idea. That said, I certainly was hoping for feedback, either to create a better proposal, or at least learn something. Whomever is willing to discuss this with me, I am willing to listen, and or learn. When no one wants to discuss this, I will not labor the point.
I feel like I have failed in communicating some of my ideas and use cases to you. In the case you bring up about tracking the history I could indeed handle tracking in the manor you described, if I was in control of all the code in use. If however, I created a metavar in one of my own applications, and passed it down into some preexisting image processing pipeline, there would be no way for me to control the behavior. WIth my proposal, I get to define a variable that can meet my needs as well as those of the library I may be using.
Another example is template expressions from my examples.py (which may be a bad name, as in c++ they are templates, even though the concept is the same, the names dont carry over well) By creating a proxy object in an arithmetic method call, arithmetic operations can be deferred and tracked inside the proxies. This can be done any number of times, with the final proxy object being bound to the variable name, i.e.
y = x1+ {<= this creates a proxy object, also with add defined} x2+x3+x4
When y is accessed __getself__ would be called, which could evaluate the stored expressions in a single loop and returned. This saves creating temporary object and potentially many expensive python for loops. (The use case I had mainly in mind is something like an array, but any object could make use of this)
Another use case is that of defining const. With suitable definitions of __setself__ and __getself__ a variable could be wrapped that could then not be rebound. Many modules make use of constants, and it may be advantageous to protect these through enforcing const-ness. You could put them behind a class with __setattr__ set, but that only changes the immutability of the class instance, leaving the instance free to be re-bound. I know the Pybind11 people struggle with how best to track and account for the lack of const-ness in python. It would be a fair point to say python should not try to cater to other projects, but it would not hurt if it was a bonus. I know even just sticking to pure python, the software stack I work on has a sufficiently large number of moving pieces, which end user consumers to worry about, we would be very happy to have a little extra safely from const.
Overall I am happy to have had feed back from everyone, and am grateful for the time that they have put in. If nothing else I have learned about what it takes to communicate ideas to a large number of people who are not co-located.