Re: [Python-ideas] A better (simpler) approach to PEP 505
Interesting, looking into, let's say, 100 of those is [not] None, randomly chosen, should give an very good idea of typical use for your code base. I am curious about why you don't like solutions involving exceptions? Performance? Catching too much, like typing errors? Less easy to distinguish between None and attr missing? Less easy to combine None-aware and classic attr lookup on the same expression? Something else?
BTW, I did (very quickly, so it's rough, my regexps are not really catching everything) the same or our code base: "is None"+ "is not None": 5600 AttributeError: 160 3 args getattr: 60 hasattr: 1800 So very similar to your pattern....except for hasattr, which is much more common in my case...
Le 25/07/2018 à 08:49, Grégory Lielens a écrit : the use cases and statistics) I think the use case here is not really the simple 'is None' + 'is not None'. if the case is just to replace one None value by something else, it's not really an improvement as we just save one short line, and once the new syntax is accepted and we're used to it, both solution are quite explicit with what they do, so I'd prefer the status quo over a second way of doing the same thing. It gets interesting in json-like cases, when we traverse a deep tree in which we might encounter None at multiple levels. To me, the new functionality is better by than the status quo as it shortens the code drastically and makes it more readable (you may see the entire traversal at once). So for the statistics, what's getting interesting is knowing when we are in that second case (but of course, it's harder to get them using simple regexes). Maybe we could find when there are more than one "is None" or "is not None" within the small block of code, and then manually check a few random dozens of those cases to see how many of them would be improved? Another thing that would be interesting in those statistics, but probably even harder to get automatically, is the proportion of cases where there is a side effect involved (so the cases when we also need a temporary variable not to have the side effect executed twice). Those cases benefit more of the new syntax than from the old one, until there's a good solution with ':=' or a lazy evaluation syntax, or some function return caching from the caller side. Also, it would be nice to know if those use cases come from specific libraries/functionalities (like json tree traversal or use of ORM) or from many different situations. In the first case, the improvement would probably belong inside the libraries themselves, or helpers could be created for this very purpose, and in the second case, if the problem is quite common, it gets interesting finding a solution inside Python itself, like what's described in this PEP. I didn't automate those search in my codebase, I just searched for 'is None' and looked if it was complicated enough to justify a new syntax, and over 937 'is None', 43 were close to another 'is None', so I looked them up manually, and 13 could have benefited from the proposal, in the sense I described sooner, so more than 1 'is None' removed in a single line of code, and/or there was a side effect (I needed a temp variable not to have the side effect repeated). But I didn't check for side effects in the other 894 'is None', so this number should probably be a bit bigger. Side note: we're not big JSON users, and when we do use some, we catch exceptions as it's never (here) normal not have well formed json strings, so the code never continues and we never have to use default values given by the backend.
On Wednesday, July 25, 2018 at 10:33:37 AM UTC+2, Brice Parent wrote:
I think the use case here is not really the simple 'is None' + 'is not None'.
Sure, that's why I also proposed to manually check a non-too-small samples of the None-testing occurences found by Guido . You did it on your sample, and your findings findings are what I roughly expected. I didn't do it. Well, not true, I sort of did it ;-), but I was lazy so I did not look enough to be representative, it was only a quick look at a very few instances: I did not encounter deep hierarchy descent (I think the few we have use hasattr), but I found a couple where the proposed syntax would help but just a little: it's short and would use single ?? or ??= By far, the most common case was using None as a marker for "we need a default sensible in the context", most of the time for a default argument. ?? and ??= indeed makes things slightly shorter and usually slightly clearer, but the improvement is very small as things are short and clear already. That's also why I was interested in the default argument delegating: In quite a few cases, the None default was because the real default was in a subfunction, and a way to delegate would be much more useful there. if the case is just to replace one None value by something else, it's
Yep, ditto. That's why I am -1 on ?? and ??= : They are useful, but not enough imho It gets interesting in json-like cases, when we traverse a deep tree in
which we might encounter None at multiple levels.
I hoped the PEP writers would submit such reallife examples, I think that's the motivation behind the PEP. But either I missed it, or they didn't show it yet.
BTW, I did (very quickly, so it's rough, my regexps are not really catching everything) the same or our code base: "is None"+ "is not None": 5600 AttributeError: 160 3 args getattr: 60 hasattr: 1800 So very similar to your pattern....except for hasattr, which is much more common in my case...
Le 25/07/2018 à 08:49, Grégory Lielens a écrit : the use cases and statistics) I think the use case here is not really the simple 'is None' + 'is not None'. if the case is just to replace one None value by something else, it's not really an improvement as we just save one short line, and once the new syntax is accepted and we're used to it, both solution are quite explicit with what they do, so I'd prefer the status quo over a second way of doing the same thing. It gets interesting in json-like cases, when we traverse a deep tree in which we might encounter None at multiple levels. To me, the new functionality is better by than the status quo as it shortens the code drastically and makes it more readable (you may see the entire traversal at once). So for the statistics, what's getting interesting is knowing when we are in that second case (but of course, it's harder to get them using simple regexes). Maybe we could find when there are more than one "is None" or "is not None" within the small block of code, and then manually check a few random dozens of those cases to see how many of them would be improved? Another thing that would be interesting in those statistics, but probably even harder to get automatically, is the proportion of cases where there is a side effect involved (so the cases when we also need a temporary variable not to have the side effect executed twice). Those cases benefit more of the new syntax than from the old one, until there's a good solution with ':=' or a lazy evaluation syntax, or some function return caching from the caller side. Also, it would be nice to know if those use cases come from specific libraries/functionalities (like json tree traversal or use of ORM) or from many different situations. In the first case, the improvement would probably belong inside the libraries themselves, or helpers could be created for this very purpose, and in the second case, if the problem is quite common, it gets interesting finding a solution inside Python itself, like what's described in this PEP. I didn't automate those search in my codebase, I just searched for 'is None' and looked if it was complicated enough to justify a new syntax, and over 937 'is None', 43 were close to another 'is None', so I looked them up manually, and 13 could have benefited from the proposal, in the sense I described sooner, so more than 1 'is None' removed in a single line of code, and/or there was a side effect (I needed a temp variable not to have the side effect repeated). But I didn't check for side effects in the other 894 'is None', so this number should probably be a bit bigger. Side note: we're not big JSON users, and when we do use some, we catch exceptions as it's never (here) normal not have well formed json strings, so the code never continues and we never have to use default values given by the backend.
On Wednesday, July 25, 2018 at 10:33:37 AM UTC+2, Brice Parent wrote:
I think the use case here is not really the simple 'is None' + 'is not None'.
Sure, that's why I also proposed to manually check a non-too-small samples of the None-testing occurences found by Guido . You did it on your sample, and your findings findings are what I roughly expected. I didn't do it. Well, not true, I sort of did it ;-), but I was lazy so I did not look enough to be representative, it was only a quick look at a very few instances: I did not encounter deep hierarchy descent (I think the few we have use hasattr), but I found a couple where the proposed syntax would help but just a little: it's short and would use single ?? or ??= By far, the most common case was using None as a marker for "we need a default sensible in the context", most of the time for a default argument. ?? and ??= indeed makes things slightly shorter and usually slightly clearer, but the improvement is very small as things are short and clear already. That's also why I was interested in the default argument delegating: In quite a few cases, the None default was because the real default was in a subfunction, and a way to delegate would be much more useful there. if the case is just to replace one None value by something else, it's
Yep, ditto. That's why I am -1 on ?? and ??= : They are useful, but not enough imho It gets interesting in json-like cases, when we traverse a deep tree in
which we might encounter None at multiple levels.
I hoped the PEP writers would submit such reallife examples, I think that's the motivation behind the PEP. But either I missed it, or they didn't show it yet.
participants (2)
-
Brice Parent -
Grégory Lielens