Other library code transformations

While we're eliminating uses of the string and types modules, how about other code clean-ups and modernization: d.has_key(k) --> k in d if k in d.keys() --> if k in d obj.__dict__ --> vars(obj) class X(UserList) --> class X(list) class X(UserDict) --> class X(dict) # and remove .data references 0L --> 0 1L --> 1 lambda x, y=y: fun(x,y) --> lambda x: fun(x,y) x = y.__class__ --> x is type(y) == None --> is None if item in astring --> if item in adict This is from my own list of code updates, the python library already in better shape. Raymond Hettinger

While we're eliminating uses of the string and types modules, how about other code clean-ups and modernization:
In general I'd only do these when you're working on a module anyway. And again stay out of the larger packages altogether.
d.has_key(k) --> k in d if k in d.keys() --> if k in d
OK.
obj.__dict__ --> vars(obj)
No. vars() exists in the first place to be called without arguments, to look in the interactive namespace.
class X(UserList) --> class X(list)
That would be a huge change in semantics.
class X(UserDict) --> class X(dict) # and remove .data references
Ditto.
0L --> 0 1L --> 1
Careful -- there are still semantic differences (e.g. 1<<100 == 0, 1L<<100 == 2**100).
lambda x, y=y: fun(x,y) --> lambda x: fun(x,y)
This changes semantics! If the outer y is later assigned to, the nested-scopes form tracks that change, the default arg doesn't.
x = y.__class__ --> x is type(y)
These are not the same! A metaclass can override __class__ but not type(); also, for a classic class instance type() and __class__ have different values.
== None --> is None
Yes!
if item in astring --> if item in adict
Huh? What does this mean?
This is from my own list of code updates, the python library already in better shape.
I'd rather see effort go towards other things. Have you looked at IDLEFORK yet? --Guido van Rossum (home page: http://www.python.org/~guido/)

]Raymond Hettinger]
if item in astring --> if item in adict
[Guido]
Huh? What does this mean?
I assume the only point is speed. Provided there are no collisions, "in" on a character-keyed dict acting as a set is faster than "in" applied to a string, even if the character searched for is the first character in the target string. But it's only a little faster then today. It's about 50% faster by the time you have to search ~= 50 characters into a string before finding a hit. However, most uses of "character in string" have very short "string" parts (like "ch in '/\\'"), and I don't judge the minor speed gain there worth the extra bother and obscurity of maintaining a static set (dict).

Raymond Hettinger wrote:
dont' forget: import stat; os.stat("foo")[stat.ST_MTIME] --> os.stat("foo").st_mtime But to be able to remove "import stat" everywhere the remaining functions in stat.py would have to be implemented as methods. And what about the remaining constants defined in stat.py? Bye, Walter Dörwald

I see no reason to want to deprecate the stat module, only the indexing constants in the stat tuple. --Guido van Rossum (home page: http://www.python.org/~guido/)

Raymond Hettinger wrote:
dont' forget: import stat; os.stat("foo")[stat.ST_MTIME] --> os.stat("foo").st_mtime But to be able to remove "import stat" everywhere the remaining functions in stat.py would have to be implemented as methods. And what about the remaining constants defined in stat.py? Bye, Walter Dörwald

Walter Dörwald wrote:
While you're at it: could you also write up all these little "code cleanups" in some file so that Andrew can integrate them in the migration guide ?! Thanks, -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/

Raymond Hettinger wrote:
Great. Thanks, -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/

Raymond, while you documenting the various changes, please include a Python version number with all of them, so that the migration guide can use this information as well. Thanks, -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/

What about "foo = foo + 1" => "foo += 1"?
I'm not for making peephole changes like this. It's easy to make mistakes (even if you run the test suite) if you don't guess the type of a variable right. I think it's better to bring code up to date in style only as part of a serious rewrite of the module containing it -- so you can fix up all different aspects. It's often kind of strange to see a modernization like this in code that otherwise shows it hasn't been modified in 5 years... (Exceptions are to get rid of deprecation warnings, or outright failures, of course.) --Guido van Rossum (home page: http://www.python.org/~guido/)

"GvR" == Guido van Rossum <guido@python.org> writes:
>> What about "foo = foo + 1" => "foo += 1"? GvR> I'm not for making peephole changes like this. It's easy to GvR> make mistakes (even if you run the test suite) if you don't GvR> guess the type of a variable right. I think it's better to GvR> bring code up to date in style only as part of a serious GvR> rewrite of the module containing it -- so you can fix up all GvR> different aspects. It's often kind of strange to see a GvR> modernization like this in code that otherwise shows it GvR> hasn't been modified in 5 years... I agree! If it works, don't fix it. I was responding to this > While you're at it: could you also write up all these little > "code cleanups" in some file so that Andrew can integrate them > in the migration guide ?! I think it's a nice "code cleanup" that is worth noting in a migration guide. -Barry

I think MAL wrote that. I interpreted it as "Raymond should document which modules he changed, and how, so that people can be aware of the subtle semantic changes." I now realize that he probably meant "can you write up a list of things you can do to modernize your code." But IMO the latter doesn't belong in a migration guide; the migration guide should focus on what you *have* to change in order to avoid disappointments later. Most of the things Raymond does aren't about new features in 2.3 either. And I *do* think that a migration guide should at least contain a general warning about the kind of changes that Raymond did that might affect 3rd party code. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
I meant the latter. This information is needed in order to moderinise Python code and also to have a reference for checking existing code against a specific Python version.
I know. That's why I asked Raymond to add a Python version to each of the modifications (basically pointing out in which Python version this coding style became available). Note that migration does not only include correcting code which might break; it also covers code cleanups like what Raymond is currently doing. I somehow have a feeling that you are afraid of such a guide, Guido. Is that so ? and if yes, why ? I think all this is valuable information and worth publishing.
Certainly. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/

No, not at all! I just misunderstood what your purpose of a migration guide was. Sorry for the confusion. --Guido van Rossum (home page: http://www.python.org/~guido/)

[Guido van Rossum]
No, not at all! I just misunderstood what your purpose of a migration guide was. Sorry for the confusion.
Certainly distinguishing between Required changes and Recommended changes would be a good thing. Required changes are what needs to be done to keep old code working. Recommended changes could include stylistic changes, new features, new idioms, etc. I think it would even make sense to talk about changes that should be made in anticipation of future feature deprecation. --- Patrick K. O'Brien Orbtech

I'm not sure that using x+=1 instead of x=x+1 should be even a recommended change. This is a personal choice, just like using True/False to indicate truth values. The "is None" vs. "== None" issue is a general style recommendation, not a migration tip. This is a "should do" issue. The "if not x" vs. "if x is None" issue is also a general style recommendation. This is a "could do" issue, because the semantics are different. --Guido van Rossum (home page: http://www.python.org/~guido/)

[Guido van Rossum]
But the choice only becomes available with a certain version of Python.
The "is None" vs. "== None" issue is a general style recommendation, not a migration tip. This is a "should do" issue.
Right. But it wouldn't hurt to remind people in a migration guide, would it?
Perhaps three sections then: Required, Recommended and Optional (TMTOWTDI)? I just think it would be good to know what coding changes one could/should start making when a new version is released. And which practices one should stop. And if there are lots of examples in real code of certain poor practices, it wouldn't hurt to point them out as well, even if they aren't necessarily tied to a particular release of Python. (But I'm willing to concede that I might be stretching the scope of this migration guide beyond its limits with that last item.) --- Patrick K. O'Brien Orbtech

2.0.
Yes it would. A migration guide should focus on migration and leave general style tips to other documents. --Guido van Rossum (home page: http://www.python.org/~guido/)

While we're eliminating uses of the string and types modules, how about other code clean-ups and modernization:
In general I'd only do these when you're working on a module anyway. And again stay out of the larger packages altogether.
d.has_key(k) --> k in d if k in d.keys() --> if k in d
OK.
obj.__dict__ --> vars(obj)
No. vars() exists in the first place to be called without arguments, to look in the interactive namespace.
class X(UserList) --> class X(list)
That would be a huge change in semantics.
class X(UserDict) --> class X(dict) # and remove .data references
Ditto.
0L --> 0 1L --> 1
Careful -- there are still semantic differences (e.g. 1<<100 == 0, 1L<<100 == 2**100).
lambda x, y=y: fun(x,y) --> lambda x: fun(x,y)
This changes semantics! If the outer y is later assigned to, the nested-scopes form tracks that change, the default arg doesn't.
x = y.__class__ --> x is type(y)
These are not the same! A metaclass can override __class__ but not type(); also, for a classic class instance type() and __class__ have different values.
== None --> is None
Yes!
if item in astring --> if item in adict
Huh? What does this mean?
This is from my own list of code updates, the python library already in better shape.
I'd rather see effort go towards other things. Have you looked at IDLEFORK yet? --Guido van Rossum (home page: http://www.python.org/~guido/)

]Raymond Hettinger]
if item in astring --> if item in adict
[Guido]
Huh? What does this mean?
I assume the only point is speed. Provided there are no collisions, "in" on a character-keyed dict acting as a set is faster than "in" applied to a string, even if the character searched for is the first character in the target string. But it's only a little faster then today. It's about 50% faster by the time you have to search ~= 50 characters into a string before finding a hit. However, most uses of "character in string" have very short "string" parts (like "ch in '/\\'"), and I don't judge the minor speed gain there worth the extra bother and obscurity of maintaining a static set (dict).

Raymond Hettinger wrote:
dont' forget: import stat; os.stat("foo")[stat.ST_MTIME] --> os.stat("foo").st_mtime But to be able to remove "import stat" everywhere the remaining functions in stat.py would have to be implemented as methods. And what about the remaining constants defined in stat.py? Bye, Walter Dörwald

I see no reason to want to deprecate the stat module, only the indexing constants in the stat tuple. --Guido van Rossum (home page: http://www.python.org/~guido/)

Raymond Hettinger wrote:
dont' forget: import stat; os.stat("foo")[stat.ST_MTIME] --> os.stat("foo").st_mtime But to be able to remove "import stat" everywhere the remaining functions in stat.py would have to be implemented as methods. And what about the remaining constants defined in stat.py? Bye, Walter Dörwald

Walter Dörwald wrote:
While you're at it: could you also write up all these little "code cleanups" in some file so that Andrew can integrate them in the migration guide ?! Thanks, -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/

Raymond Hettinger wrote:
Great. Thanks, -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/

Raymond, while you documenting the various changes, please include a Python version number with all of them, so that the migration guide can use this information as well. Thanks, -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/

What about "foo = foo + 1" => "foo += 1"?
I'm not for making peephole changes like this. It's easy to make mistakes (even if you run the test suite) if you don't guess the type of a variable right. I think it's better to bring code up to date in style only as part of a serious rewrite of the module containing it -- so you can fix up all different aspects. It's often kind of strange to see a modernization like this in code that otherwise shows it hasn't been modified in 5 years... (Exceptions are to get rid of deprecation warnings, or outright failures, of course.) --Guido van Rossum (home page: http://www.python.org/~guido/)

"GvR" == Guido van Rossum <guido@python.org> writes:
>> What about "foo = foo + 1" => "foo += 1"? GvR> I'm not for making peephole changes like this. It's easy to GvR> make mistakes (even if you run the test suite) if you don't GvR> guess the type of a variable right. I think it's better to GvR> bring code up to date in style only as part of a serious GvR> rewrite of the module containing it -- so you can fix up all GvR> different aspects. It's often kind of strange to see a GvR> modernization like this in code that otherwise shows it GvR> hasn't been modified in 5 years... I agree! If it works, don't fix it. I was responding to this > While you're at it: could you also write up all these little > "code cleanups" in some file so that Andrew can integrate them > in the migration guide ?! I think it's a nice "code cleanup" that is worth noting in a migration guide. -Barry

I think MAL wrote that. I interpreted it as "Raymond should document which modules he changed, and how, so that people can be aware of the subtle semantic changes." I now realize that he probably meant "can you write up a list of things you can do to modernize your code." But IMO the latter doesn't belong in a migration guide; the migration guide should focus on what you *have* to change in order to avoid disappointments later. Most of the things Raymond does aren't about new features in 2.3 either. And I *do* think that a migration guide should at least contain a general warning about the kind of changes that Raymond did that might affect 3rd party code. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
I meant the latter. This information is needed in order to moderinise Python code and also to have a reference for checking existing code against a specific Python version.
I know. That's why I asked Raymond to add a Python version to each of the modifications (basically pointing out in which Python version this coding style became available). Note that migration does not only include correcting code which might break; it also covers code cleanups like what Raymond is currently doing. I somehow have a feeling that you are afraid of such a guide, Guido. Is that so ? and if yes, why ? I think all this is valuable information and worth publishing.
Certainly. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/

No, not at all! I just misunderstood what your purpose of a migration guide was. Sorry for the confusion. --Guido van Rossum (home page: http://www.python.org/~guido/)

[Guido van Rossum]
No, not at all! I just misunderstood what your purpose of a migration guide was. Sorry for the confusion.
Certainly distinguishing between Required changes and Recommended changes would be a good thing. Required changes are what needs to be done to keep old code working. Recommended changes could include stylistic changes, new features, new idioms, etc. I think it would even make sense to talk about changes that should be made in anticipation of future feature deprecation. --- Patrick K. O'Brien Orbtech

I'm not sure that using x+=1 instead of x=x+1 should be even a recommended change. This is a personal choice, just like using True/False to indicate truth values. The "is None" vs. "== None" issue is a general style recommendation, not a migration tip. This is a "should do" issue. The "if not x" vs. "if x is None" issue is also a general style recommendation. This is a "could do" issue, because the semantics are different. --Guido van Rossum (home page: http://www.python.org/~guido/)

[Guido van Rossum]
But the choice only becomes available with a certain version of Python.
The "is None" vs. "== None" issue is a general style recommendation, not a migration tip. This is a "should do" issue.
Right. But it wouldn't hurt to remind people in a migration guide, would it?
Perhaps three sections then: Required, Recommended and Optional (TMTOWTDI)? I just think it would be good to know what coding changes one could/should start making when a new version is released. And which practices one should stop. And if there are lots of examples in real code of certain poor practices, it wouldn't hurt to point them out as well, even if they aren't necessarily tied to a particular release of Python. (But I'm willing to concede that I might be stretching the scope of this migration guide beyond its limits with that last item.) --- Patrick K. O'Brien Orbtech

2.0.
Yes it would. A migration guide should focus on migration and leave general style tips to other documents. --Guido van Rossum (home page: http://www.python.org/~guido/)
participants (8)
-
barry@zope.com
-
Fredrik Lundh
-
Guido van Rossum
-
M.-A. Lemburg
-
Patrick K. O'Brien
-
Raymond Hettinger
-
Tim Peters
-
Walter Dörwald