deprecating string module?
I know this has been discussed before. In light of the recent PEP8 update: Avoid the use of the string module; instead use string methods. These are always much faster and share the same API with unicode strings. I thought I'd raise it again. How about deprecating the string module? In the past, this has presented two major problems. First, its near ubiquity in Python programs before the availability of string methods. Second, the presence of a few useful data objects (digits, uppercase, etc). The first problem can be solved by extending the deprecation time suitably (two years/four releases?) I think the second problem can be solved by adding those data objects to either sys or locale(preferably the latter). string.letters and its subsets are locale-dependent. In relevant locales I presume string.punctuation and string.whitespace would also be different than their common ASCII elements. Finally, I'd amend the above PEP8 admontion something like Avoid the use of the string module unless backward-compatibility with versions earlier than Python 2.0 is important; instead use string methods. These are always much faster and share the same API with unicode strings. Skip
Skip Montanaro <skip@pobox.com>:
the presence of a few useful data objects (digits, uppercase, etc). ... I think the second problem can be solved by adding those data objects to either sys or locale(preferably the latter).
Making them class attributes of str has also been suggested. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
Hi, Skip Montanaro wrote:
I know this has been discussed before. In light of the recent PEP8 update:
Avoid the use of the string module; instead use string methods. These are always much faster and share the same API with unicode strings.
I thought I'd raise it again.
How about deprecating the string module?
I'm absolutely against depreceating the string module! Instead I suggest to change the cited wording from PEP8 into Avoid the use of the string module unless you want to keep compatibility with Python 1.5.2; instead use the string methods introduced in later releases of Python. These are always much faster and share the same API with unicode strings. Python 1.5.2 is still very important and even the latest Red Hat distributions still use it. Since the bytecode (.pyc) has changed inbetween, we decided to ship all our Python Software with 1.5.2 and backport useful new library modules from 2.x to 1.5.2 if we want to use them. Of course the edits needed to turn string method calls back into string module function calls for 1.5.2 are trivial but they are nevertheless tedious. Very often the overall performance win of using string methods will be so negligible, that I tend to believe that the CPU cycles saved due to this change don't sum up to the amount of programmer time wasted by editing Python software from string function calls into string method calls or vice versa. Regards, Peter -- Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260 office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen, Germany)
Peter Funk wrote:
Hi,
Skip Montanaro wrote:
I know this has been discussed before. In light of the recent PEP8 update:
Avoid the use of the string module; instead use string methods. These are always much faster and share the same API with unicode strings.
I thought I'd raise it again.
How about deprecating the string module?
I'm absolutely against depreceating the string module! Instead I suggest to change the cited wording from PEP8 into
Avoid the use of the string module unless you want to keep compatibility with Python 1.5.2; instead use the string methods introduced in later releases of Python. These are always much faster and share the same API with unicode strings.
Python 1.5.2 is still very important and even the latest Red Hat distributions still use it. Since the bytecode (.pyc) has changed inbetween, we decided to ship all our Python Software with 1.5.2 and backport useful new library modules from 2.x to 1.5.2 if we want to use them.
Of course the edits needed to turn string method calls back into string module function calls for 1.5.2 are trivial but they are nevertheless tedious. Very often the overall performance win of using string methods will be so negligible, that I tend to believe that the CPU cycles saved due to this change don't sum up to the amount of programmer time wasted by editing Python software from string function calls into string method calls or vice versa.
There's a good example of what I was talking about in the "Stability and Change" thread: careless use of deprecations would cause serious grief among developers and certainly also users (who are then confronted with tons of warnings which they don't understand). Now for Python modules the situation is a little better, since it is possible to wrap up those deprecated modules in a distutils package for users to install on top of their Python install. However, when touching core features of the language things become completely unmanageable. The reason is that various extensions out there start to rely on features which are only available in later releases while others are not migrated yet to these versions and would result in endless streams of deprecation warnings or even failures. As a result you'd lose the ability to fish in the sea of Python extensions... BTW, I don't see the tradeoff in saving 11k worth of diskspace for string.py compared to the good vibes we lose in the Python world for this. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/
mal> There's a good example of what I was talking about in the mal> "Stability and Change" thread: careless use of deprecations would mal> cause serious grief among developers and certainly also users (who mal> are then confronted with tons of warnings which they don't mal> understand). I don't believe I said "let's carelessly deprecate the string module". mal> Now for Python modules the situation is a little better, since it mal> is possible to wrap up those deprecated modules in a distutils mal> package for users to install on top of their Python install. I don't see why this can't be done for string.py. mal> BTW, I don't see the tradeoff in saving 11k worth of diskspace for mal> string.py compared to the good vibes we lose in the Python world mal> for this. It has nothing to do with 11k worth of disk space and everything to do with "there's one (best) way to do it". Skip
Skip Montanaro wrote:
mal> There's a good example of what I was talking about in the mal> "Stability and Change" thread: careless use of deprecations would mal> cause serious grief among developers and certainly also users (who mal> are then confronted with tons of warnings which they don't mal> understand).
I don't believe I said "let's carelessly deprecate the string module".
Looking at the subject line, you're almost there ;-)
mal> Now for Python modules the situation is a little better, since it mal> is possible to wrap up those deprecated modules in a distutils mal> package for users to install on top of their Python install.
I don't see why this can't be done for string.py.
Sure it can; don't see what we gain, though. Note that I was just addressing the more general case discussed in the other thread "Stability and change".
mal> BTW, I don't see the tradeoff in saving 11k worth of diskspace for mal> string.py compared to the good vibes we lose in the Python world mal> for this.
It has nothing to do with 11k worth of disk space and everything to do with "there's one (best) way to do it".
That's a silly phrase. I agree that there should always be a straight-forward and intuitive way to do things, but I don't go for such a dogma type argument and I'm sure you don't either :-) The intuitive way to do e.g. string.find has changed from import string string.find(x,y) to x.find(y) and that's worth making clear, but forcing the Python people to change their previously adapted intuition sounds too much like dictatorship to me. If I want to write code which works in all Python versions starting from 1.5.2 onwards, I should be able to do this. Deprecating important modules like string and types and then removing them altogether makes this impossible without extra hackery or providing special bwcompat packages to the users. Just look at the hoops which the email package has to go through to maintain such compatibility. That's not the old Python philosophy I'm used to and that's also why I dislike the idea to rip out code for no apparently needed reason at all. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/
>> It has nothing to do with 11k worth of disk space and everything to >> do with "there's one (best) way to do it". mal> That's a silly phrase. I agree that there should always be a mal> straight-forward and intuitive way to do things, but I don't go for mal> such a dogma type argument and I'm sure you don't either :-) It's not dogma. It's a catch phrase, I agree, but I think the concept does have merit. mal> If I want to write code which works in all Python versions starting mal> from 1.5.2 onwards, I should be able to do this. This seems like a good project for the Python Business Forum. How about developing some hard numbers that indicate what fraction of Python users use what version of Python? It would have to be sensitive to the type of users and their platform, but would be useful information for all Python programmers to have. I am personally getting rather tired of the "you can't do that! it's not 1.5.2-compatible!" whine without any justification other than "RedHat still ships 1.5.2". -- Skip Montanaro (skip@pobox.com - http://www.mojam.com/) Boycott Netflix - they spam
Skip Montanaro wrote:
>> It has nothing to do with 11k worth of disk space and everything to >> do with "there's one (best) way to do it".
mal> That's a silly phrase. I agree that there should always be a mal> straight-forward and intuitive way to do things, but I don't go for mal> such a dogma type argument and I'm sure you don't either :-)
It's not dogma. It's a catch phrase, I agree, but I think the concept does have merit.
Sure it does, but not in the sense of "there's only one way to do it: mine" ;-)
mal> If I want to write code which works in all Python versions starting mal> from 1.5.2 onwards, I should be able to do this.
This seems like a good project for the Python Business Forum. How about developing some hard numbers that indicate what fraction of Python users use what version of Python? It would have to be sensitive to the type of users and their platform, but would be useful information for all Python programmers to have.
Indeed.
I am personally getting rather tired of the "you can't do that! it's not 1.5.2-compatible!" whine without any justification other than "RedHat still ships 1.5.2".
eGenix and most other companies providing Python extensions I know of still support Python 1.5.2. Many users still like 1.5.2 for its stability and better performance. For some statistics, I get around 4000 hits on Google for "Python 1.4" 52000 hits on Google for "Python 1.5" 31000 hits on Google for "Python 2.0" 44000 hits on Google for "Python 2.1" 36000 hits on Google for "Python 2.2" 2000 hits on Google for "Python 2.3" From this I take that the world has moved away from 1.4 but we're still far from saying Python 1.5.2 is dead. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/
On Wed, May 29, 2002, Skip Montanaro wrote:
I am personally getting rather tired of the "you can't do that! it's not 1.5.2-compatible!" whine without any justification other than "RedHat still ships 1.5.2".
So don't keep proposing changes that break 1.5.2 compatibility. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "In the end, outside of spy agencies, people are far too trusting and willing to help." --Ira Winkler
So don't keep proposing changes that break 1.5.2 compatibility.
-1. That means we could never deprecate a feature. --Guido van Rossum (home page: http://www.python.org/~guido/)
On Wed, May 29, 2002, Guido van Rossum wrote:
Aahz:
So don't keep proposing changes that break 1.5.2 compatibility.
-1. That means we could never deprecate a feature.
In case it wasn't clear, that was sarcasm in response to Skip's "tired of whining" comment. I like to think I occupy the middle ground on the backward compatibility issue, and at various times I like to take potshots at both extremes. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "In the end, outside of spy agencies, people are far too trusting and willing to help." --Ira Winkler
Aahz to Skip:
So don't keep proposing changes that break 1.5.2 compatibility.
Guido van Rossum:
-1. That means we could never deprecate a feature.
Than don't do it! Simply never remove features from the language (whether the feature is documented or not). Do it like Niklaus Wirth: Always start with a fresh new name for the language, whenever a major cleanup of unwanted features seems to be absolutely unavoidable to you: Wirth invented Pascal, later Modula-2 and finally Oberon. However ... Orlijthon ... hmmm ... this seems to be somewhat hard to type and spell. ;-) Regards, Peter -- Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260 office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen, Germany)
The numbering scheme should cover this need. After all, Wirth had a series of numbered versions for Modula (<empty>, 2, 3). I think the individual modules may need explicit version numbering, also. I find 15 of 158 .py files in my release of the library have __version__ numbers. I have a patch I use for the current release of the SGML module; it may never make it to prime time (since it works around an obscure MS-Word-ism). I may have to update it for future releases, and would like to know which releases of the module use wich version of the patch. --- Peter Funk <pf@artcom-gmbh.de> wrote:
Aahz to Skip:
So don't keep proposing changes that break 1.5.2 compatibility.
Guido van Rossum:
-1. That means we could never deprecate a feature.
Than don't do it! Simply never remove features from the language (whether the feature is documented or not).
Do it like Niklaus Wirth: Always start with a fresh new name for the language, ...
===== -- S. Lott, CCP :-{) S_LOTT@YAHOO.COM http://www.mindspring.com/~slott1 Buccaneer #468: KaDiMa Macintosh user: drinking upstream from the herd. __________________________________________________ Do You Yahoo!? Yahoo! - Official partner of 2002 FIFA World Cup http://fifaworldcup.yahoo.com
Steven Lott wrote:
The numbering scheme should cover this need. After all, Wirth had a series of numbered versions for Modula (<empty>, 2, 3).
I think the individual modules may need explicit version numbering, also. I find 15 of 158 .py files in my release of the library have __version__ numbers.
IMO Enforcing version numbers in standard libraries is a good idea. Eventually, a good versioning scheme for the standard-lib with automated deprecation may be what is needed. Making the introduction of new (or deprecation of) features a) by hand b) by measures of 'x years', example from from PEP4: "The deprecation warning will be added to the module one year after Python 2.3 is released, and the module will be removed one year after that." seems unreliable. How do you construct an if-statement for 'One year after 2.3 is released' if the current version is 2.2 <wink>. Instead deprecating a) (semi-) automatically b) by saying '2.3 issues deprecation warning, 2.4 does not support it'. seems much saner. Even a program can be teached to check this. The more intransparent the versioning/deprecation scheme the harder it is to handle and the more people will scream at every deprecation-proposal. deprecated-4223-seconds-after-7th-reply'ly yours, holger
This would be very cool. Rather than having to go by python version numbers, which seem obscure, an application can declare its dependencies by module. Perhaps even some tool to determine an apps dependencies. These dependencies can then be checked using the current version in a perl-esque regression test style to determine how well the current version meets the applications needs (I say this because some code may not be run normally but require more advanced features - this could also allow for some very interesting approaches to modularity and using available python features in larger applications). It would then be very easy to determine the cause of breakage and/or the need of the application. -- Mike On Wed, May 29 @ 20:29, holger krekel wrote:
IMO Enforcing version numbers in standard libraries is a good idea.
Eventually, a good versioning scheme for the standard-lib with automated deprecation may be what is needed. Making the introduction of new (or deprecation of) features
a) by hand
b) by measures of 'x years', example from from PEP4: "The deprecation warning will be added to the module one year after Python 2.3 is released, and the module will be removed one year after that."
seems unreliable. How do you construct an if-statement for 'One year after 2.3 is released' if the current version is 2.2 <wink>.
Instead deprecating
a) (semi-) automatically
b) by saying '2.3 issues deprecation warning, 2.4 does not support it'.
seems much saner. Even a program can be teached to check this.
The more intransparent the versioning/deprecation scheme the harder it is to handle and the more people will scream at every deprecation-proposal.
deprecated-4223-seconds-after-7th-reply'ly yours, holger
-- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html
This would be very cool. Rather than having to go by python version numbers, which seem obscure, an application can declare its dependencies by module. Perhaps even some tool to determine an apps dependencies. These dependencies can then be checked using the current version in a perl-esque regression test style to determine how well the current version meets the applications needs (I say this because some code may not be run normally but require more advanced features - this could also allow for some very interesting approaches to modularity and using available python features in larger applications). It would then be very easy to determine the cause of breakage and/or the need of the application.
Hm, this sounds like overkill to me. And who's going to write the AI software to do this automatic regression testing? --Guido van Rossum (home page: http://www.python.org/~guido/)
On Thu, May 30 @ 12:31, Guido van Rossum wrote:
This would be very cool. Rather than having to go by python version numbers, which seem obscure, an application can declare its dependencies by module. Perhaps even some tool to determine an apps dependencies. These dependencies can then be checked using the current version in a perl-esque regression test style to determine how well the current version meets the applications needs (I say this because some code may not be run normally but require more advanced features - this could also allow for some very interesting approaches to modularity and using available python features in larger applications). It would then be very easy to determine the cause of breakage and/or the need of the application.
Hm, this sounds like overkill to me. And who's going to write the AI software to do this automatic regression testing?
Perhaps it came across as more than it should be. I meant just specifying the version numbers of the individual modules in a CPAN-like dependency thing. Then, on start-up, the dependencies are just checked. Mostly matching of version numbers. The stats are just a fancy way of saying how screwed you are :) -- Mike -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html
This would be very cool. Rather than having to go by python version numbers, which seem obscure, an application can declare its dependencies by module. Perhaps even some tool to determine an apps dependencies. These dependencies can then be checked using the current version in a perl-esque regression test style to determine how well the current version meets the applications needs (I say this because some code may not be run normally but require more advanced features - this could also allow for some very interesting approaches to modularity and using available python features in larger applications). It would then be very easy to determine the cause of breakage and/or the need of the application.
Hm, this sounds like overkill to me. And who's going to write the AI software to do this automatic regression testing?
Perhaps it came across as more than it should be. I meant just specifying the version numbers of the individual modules in a CPAN-like dependency thing. Then, on start-up, the dependencies are just checked. Mostly matching of version numbers. The stats are just a fancy way of saying how screwed you are :)
This has been proposed before, but I don't think anybody really understands how to do version dependencies correctly. --Guido van Rossum (home page: http://www.python.org/~guido/)
Well, I'll have to think about it and get back to you :) On Thu, May 30 @ 13:06, Guido van Rossum wrote:
This would be very cool. Rather than having to go by python version numbers, which seem obscure, an application can declare its dependencies by module. Perhaps even some tool to determine an apps dependencies. These dependencies can then be checked using the current version in a perl-esque regression test style to determine how well the current version meets the applications needs (I say this because some code may not be run normally but require more advanced features - this could also allow for some very interesting approaches to modularity and using available python features in larger applications). It would then be very easy to determine the cause of breakage and/or the need of the application.
Hm, this sounds like overkill to me. And who's going to write the AI software to do this automatic regression testing?
Perhaps it came across as more than it should be. I meant just specifying the version numbers of the individual modules in a CPAN-like dependency thing. Then, on start-up, the dependencies are just checked. Mostly matching of version numbers. The stats are just a fancy way of saying how screwed you are :)
This has been proposed before, but I don't think anybody really understands how to do version dependencies correctly. `-> (guido)
-- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html
Guido van Rossum wrote:
This would be very cool. Rather than having to go by python version numbers, which seem obscure, an application can declare its dependencies by module. Perhaps even some tool to determine an apps dependencies. These dependencies can then be checked using the current version in a perl-esque regression test style to determine how well the current version meets the applications needs (I say this because some code may not be run normally but require more advanced features - this could also allow for some very interesting approaches to modularity and using available python features in larger applications). It would then be very easy to determine the cause of breakage and/or the need of the application.
Hm, this sounds like overkill to me. And who's going to write the AI software to do this automatic regression testing?
Perhaps it came across as more than it should be. I meant just specifying the version numbers of the individual modules in a CPAN-like dependency thing. Then, on start-up, the dependencies are just checked. Mostly matching of version numbers. The stats are just a fancy way of saying how screwed you are :)
This has been proposed before, but I don't think anybody really understands how to do version dependencies correctly.
But i assume, you do agree that a *working* versioning and dependency scheme for the standard-lib would be more than nice. (IMHO the more modules and versions we have the more important it gets). holger
But i assume, you do agree that a *working* versioning and dependency scheme for the standard-lib would be more than nice. (IMHO the more modules and versions we have the more important it gets).
Depends on how you define "working". I'm skeptical that such a thing can exist. --Guido van Rossum (home page: http://www.python.org/~guido/)
Michael Gilfix wrote:
This would be very cool. Rather than having to go by python version numbers, which seem obscure, an application can declare its dependencies by module. Perhaps even some tool to determine an apps dependencies. (...)
while i agree to this basic idea i wanted to remind of the other point buried in my mail ... Changing the deprecation process to be based on *version numbers* rather than *uncertain future points in time*...
On Wed, May 29 @ 20:29, holger krekel wrote:
IMO Enforcing version numbers in standard libraries is a good idea.
Eventually, a good versioning scheme for the standard-lib with automated deprecation may be what is needed. Making the introduction of new (or deprecation of) features
a) by hand
b) by measures of 'x years', example from from PEP4: "The deprecation warning will be added to the module one year after Python 2.3 is released, and the module will be removed one year after that."
seems unreliable. How do you construct an if-statement for 'One year after 2.3 is released' if the current version is 2.2 <wink>.
Instead deprecating
a) (semi-) automatically
b) by saying '2.3 issues deprecation warning, 2.4 does not support it'.
seems much saner. Even a program can be teached to check this.
Am i the only one to find this an evident change for the good (tm) ? holger
while i agree to this basic idea i wanted to remind of the other point buried in my mail ... Changing the deprecation process to be based on *version numbers* rather than *uncertain future points in time*...
-1. It's hard to predict how often versions will be released. The PEPs (4 and 5) specify time limits, which are better defined. --Guido van Rossum (home page: http://www.python.org/~guido/)
On Thu, May 30 @ 16:58, Guido van Rossum wrote:
while i agree to this basic idea i wanted to remind of the other point buried in my mail ... Changing the deprecation process to be based on *version numbers* rather than *uncertain future points in time*...
-1. It's hard to predict how often versions will be released. The PEPs (4 and 5) specify time limits, which are better defined.
The problem with this is the "relative to whom" question... Is the time limit better defined relative to the developers or users? No user should really be using bleeding edge CVS (without getting what he asked for) for any production quality thing. As far as they're concerned, version numbers are the atomic units of change. -- Mike -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html
Guido van Rossum wrote:
while i agree to this basic idea i wanted to remind of the other point buried in my mail ... Changing the deprecation process to be based on *version numbers* rather than *uncertain future points in time*...
-1. It's hard to predict how often versions will be released. The PEPs (4 and 5) specify time limits, which are better defined.
ok then. how would you translate (from PEP4) "The deprecation warning will be added to the module one year after Python 2.3 is released, and the module will be removed one year after that." to an if-statement-check you want to implement *today*? Additionally, if the above was based on versions you could implement the *deprecation process* today, no need to remember at which date some PEP mentioned something which has been forgotten for three years <wink> anyway. Are we really supposed to remember release dates of python versions? I mean of course they are of great historic value, but ... holger
while i agree to this basic idea i wanted to remind of the other point buried in my mail ... Changing the deprecation process to be
Guido van Rossum wrote: based
on *version numbers* rather than *uncertain future points in time*...
-1. It's hard to predict how often versions will be released. The PEPs (4 and 5) specify time limits, which are better defined.
[Holger Kregel]:
ok then. how would you translate (from PEP4)
"The deprecation warning will be added to the module one year after Python 2.3 is released, and the module will be removed one year after that."
to an if-statement-check you want to implement *today*?
Additionally, if the above was based on versions you could implement the *deprecation process* today, no need to remember at which date some PEP mentioned something which has been forgotten for three years <wink> anyway.
Are we really supposed to remember release dates of python versions? I mean of course they are of great historic value, but ...
Well, personally I think it would be great if every release silently suppressed deprecation warnings for code older than the release. So it would only need to know its own release date. But I've made the suggestion before, so apparently nobody thinks this is a sensible way to achieve "silent deprecation". at-least-this-thread-taught-me-how-to-spell-deprecation-ly y'rs - steve ----------------------------------------------------------------------- Steve Holden http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/pwp/ -----------------------------------------------------------------------
Well, personally I think it would be great if every release silently suppressed deprecation warnings for code older than the release. So it would only need to know its own release date. But I've made the suggestion before, so apparently nobody thinks this is a sensible way to achieve "silent deprecation".
I think it would defeat the purpose. Plus, how do you determine the age? --Guido van Rossum (home page: http://www.python.org/~guido/)
----- Original Message ----- From: "Guido van Rossum" <guido@python.org> To: "Steve Holden" <sholden@holdenweb.com> Cc: <python-dev@python.org> Sent: Thursday, May 30, 2002 9:30 PM Subject: Re: [Python-Dev] Re: Deprecation
Well, personally I think it would be great if every release silently suppressed deprecation warnings for code older than the release. So it would only need to know its own release date. But I've made the suggestion before, so apparently nobody thinks this is a sensible way to achieve "silent deprecation".
I think it would defeat the purpose. Plus, how do you determine the age?
Clearly you'd have to goby the source file date, since a new release would recompile the .pyc's. The main advantage is that deprecation wornings wouldn't start appearing after the installation of a new release. The only advantage I see for it is the silence. Obviously it would eventually mean that programs died a sudden death due to feature loss. The warnings would appear if the code were maintained, however. regards ----------------------------------------------------------------------- Steve Holden http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/pwp/ -----------------------------------------------------------------------
Clearly you'd have to goby the source file date, since a new release would recompile the .pyc's. The main advantage is that deprecation wornings wouldn't start appearing after the installation of a new release. The only advantage I see for it is the silence. Obviously it would eventually mean that programs died a sudden death due to feature loss. The warnings would appear if the code were maintained, however.
I definitely wouldn't want such a feature. --Guido van Rossum (home page: http://www.python.org/~guido/)
ok then. how would you translate (from PEP4)
"The deprecation warning will be added to the module one year after Python 2.3 is released, and the module will be removed one year after that."
to an if-statement-check you want to implement *today*?
I guess I don't see why the error message should contain an expiration date. It's enough to know that the feature will expire -- you can find more info in the docs.
Additionally, if the above was based on versions you could implement the *deprecation process* today, no need to remember at which date some PEP mentioned something which has been forgotten for three years <wink> anyway.
Define deprecation process. I don't think it's a good idea to try to automate this. I want a human to decide that a deprecated module is really ready for deletion. This involves many other factors besides the originally promised deprecation date.
Are we really supposed to remember release dates of python versions? I mean of course they are of great historic value, but ...
Why would you need to? --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
ok then. how would you translate (from PEP4)
"The deprecation warning will be added to the module one year after Python 2.3 is released, and the module will be removed one year after that."
to an if-statement-check you want to implement *today*?
I guess I don't see why the error message should contain an expiration date. It's enough to know that the feature will expire -- you can find more info in the docs.
You may want to check that the python-version your program is running on might not have a certain feature or library any more. Just seeing an 'ImportError' leaves the user puzzled if he e.g. might have an installation problem.
Additionally, if the above was based on versions you could implement the *deprecation process* today, no need to remember at which date some PEP mentioned something which has been forgotten for three years <wink> anyway.
Define deprecation process.
the citation above is a typical example for a 'deprecation process'. OK, you are a mathematician. i try harder... A deprecation process may be seen as an algorithm which determines the life time (or die-time?) of features and modules.
I don't think it's a good idea to try to automate this. I want a human to decide that a deprecated module is really ready for deletion. This involves many other factors besides the originally promised deprecation date.
automation does not mean there is no human intervention possible. It would be horrible indeed if computers would take over python development, no flaming wars, less and less more general modules ... :-) Anyway, removing a module would certainly still require human intervention but its *intended* deprecation/removal could be specified precisely. If a feature lives longer due to gods mercy <wink> than this is pure luck. You can hope for it but should not rely on it. I just happen to think that people and programs can work much better with 'version 2.5 or 3.0' than 'some years after some point in time' <wink>. time-less-ly yours, holger P.S: another solution is to found a Deprecation-SIG which would find many many friends, i bet. The only SIG that would have no expiration date :-)
I guess I don't see why the error message should contain an expiration date. It's enough to know that the feature will expire -- you can find more info in the docs.
You may want to check that the python-version your program is running on might not have a certain feature or library any more. Just seeing an 'ImportError' leaves the user puzzled if he e.g. might have an installation problem.
Yes, they hve an installation problem. :-) They are using a Python version that's incompatible with the program. I really don't see what you are proposing to do about this. The author of the program should know that it uses a deprecated module and fix the program, rather than simply continuing to distribute it with out change. If the program is very old, the user who downloads it should realize that it might no longer work.
Additionally, if the above was based on versions you could implement the *deprecation process* today, no need to remember at which date some PEP mentioned something which has been forgotten for three years <wink> anyway.
Define deprecation process.
the citation above is a typical example for a 'deprecation process'. OK, you are a mathematician. i try harder...
A deprecation process may be seen as an algorithm which determines the life time (or die-time?) of features and modules.
I think this cannot and should not be determined algorithmically.
I don't think it's a good idea to try to automate this. I want a human to decide that a deprecated module is really ready for deletion. This involves many other factors besides the originally promised deprecation date.
automation does not mean there is no human intervention possible. It would be horrible indeed if computers would take over python development, no flaming wars, less and less more general modules ... :-)
Anyway, removing a module would certainly still require human intervention but its *intended* deprecation/removal could be specified precisely. If a feature lives longer due to gods mercy <wink> than this is pure luck. You can hope for it but should not rely on it.
I just happen to think that people and programs can work much better with 'version 2.5 or 3.0' than 'some years after some point in time' <wink>.
I don't see the point. Let's end this discussion. --Guido van Rossum (home page: http://www.python.org/~guido/)
----- Original Message ----- From: "M.-A. Lemburg" <mal@lemburg.com> To: <skip@pobox.com> Cc: <python-dev@python.org> Sent: Wednesday, May 29, 2002 7:54 AM Subject: Re: [Python-Dev] deprecating string module?
Skip Montanaro wrote: [on carelessness]
mal> Now for Python modules the situation is a little better, since
it
mal> is possible to wrap up those deprecated modules in a distutils mal> package for users to install on top of their Python install.
I don't see why this can't be done for string.py.
Sure it can; don't see what we gain, though.
Well, one thing we would gain is that new users would be educated by the depracation warnings. If nslookup is allowed to tell me """Note: nslookup is deprecated and may be removed from future releases. Consider using the `dig' or `host' programs instead. Run nslookup with the `-sil[ent]' option to prevent this message from appearing.""" then it seems reasonable for Python to say "If you carry on programming like this you will (albeit eventually) come a cropper". The problem that (for instance) the Python Business Thingy might have with this is that it looks like they don't know their job if their software starts to barf all over the operating environment when a new version of Python is deployed. Their customers (or their ISPs) will report that a Python upgrade has "broken something", and that's not good PR.
Note that I was just addressing the more general case discussed in the other thread "Stability and change".
mal> BTW, I don't see the tradeoff in saving 11k worth of diskspace
for
mal> string.py compared to the good vibes we lose in the Python
world
mal> for this.
It has nothing to do with 11k worth of disk space and everything to do
with
"there's one (best) way to do it".
That's a silly phrase. I agree that there should always be a straight-forward and intuitive way to do things, but I don't go for such a dogma type argument and I'm sure you don't either :-)
"That's a silly phrase" is a silly phrase <0.36759 wink>. Dogma, shmogma. None of us buy the "there can be only one way to do it" philosophy, which clearly isn't Pythonic. Not when I'm the arbiter, anyway. While there will *always* be "more than one way to do it", I might be persuaded that a "single most obvious" way is desirable, and the more common the use cases the more desirable it is. Unless it's *really, waaay* too slow. The important thing is to cut down the repertoire of "micro design patterns" commonly used by literate Python programmers. This will tend to help the readability of actively maintained code.
The intuitive way to do e.g. string.find has changed from
import string string.find(x,y)
Clearly not for everybody <wink>.
to
x.find(y)
and that's worth making clear, but forcing the Python people to change their previously adapted intuition sounds too much like dictatorship to me.
Well, I think you're both right (sickening, isn't it). The problem is the "Python-puke" from production systems. If a script is run frequently (as a popular CGI, or similar) it's going to puke a lot, and (to the sysadmins) probably quite visibly.
If I want to write code which works in all Python versions starting from 1.5.2 onwards, I should be able to do this. Deprecating important modules like string and types and then removing them altogether makes this impossible without extra hackery or providing special bwcompat packages to the users.
I can understand this, but isn't "1.5.2" a little arbitrary. And when does "1.5.2" become something else, and should it be "2.2.3" or "3.0.7"? There will, inevitably, come a point when Python X.Y's new features offer radically more efficient and/or easily-maintained paradigms for certain tasks to the extent that maintaining 1.5.2 compatibility will be a real loss.
Just look at the hoops which the email package has to go through to maintain such compatibility. That's not the old Python philosophy I'm used to and that's also why I dislike the idea to rip out code for no apparently needed reason at all.
I understand both the words and the feelings behind this statement, and support them both. The warning system seems fine to me, but... Suppose that a feature used in existing [Python] code is depracated as a result of a minor (N.X -> N.Y). This should not affect production systems, but should definitely instigate warnings in the development environment. If the depracation warnings were deactivated *for code sourced before the release date* then 1. a commercial "product" of the Python Business Thingy would have the maximum lifetime consistent with feature support in Python, 2. developers would know for three release cycles how to maintain their code, and 3. if an unsuspecting sysadmin makes a minor edit to the code (aha, this release is being "actively maintained") then that particular module starts barfing. Of course this does have the slight downside that when feature support *is* finally removed, death is immediate and terminal. This might not be an altogether bad thing, and in a web server might be regarded as indicating time to seek the Happy Hunting Grounds. regards ----------------------------------------------------------------------- Steve Holden http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/pwp/ -----------------------------------------------------------------------
If I want to write code which works in all Python versions starting from 1.5.2 onwards, I should be able to do this. Deprecating important modules like string and types and then removing them altogether makes this impossible without extra hackery or providing special bwcompat packages to the users.
I can understand this, but isn't "1.5.2" a little arbitrary.
have you tried migrating a huge python system from 1.5.2 to 2.x? if so, what did you learn? </F>
martin wrote:
have you tried migrating a huge python system from 1.5.2 to 2.x? if so, what did you learn?
That it is easier than porting from 1.4.
this does not match my experiences (neither from 24/7 production systems, nor any of our gui applications). what kind of applications have you ported? </F>
"Fredrik Lundh" <fredrik@pythonware.com> writes:
have you tried migrating a huge python system from 1.5.2 to 2.x? if so, what did you learn?
That it is easier than porting from 1.4.
this does not match my experiences (neither from 24/7 production systems, nor any of our gui applications).
what kind of applications have you ported?
I found the primary difference in the use of the regex module: Python 1.5 code would use the re module, which was not available in 1.4. This now causes a DeprecationWarning. The most recent instance where I had to change this was YAPPS. Regards, Martin
"MAL" == M <mal@lemburg.com> writes:
MAL> Just look at the hoops which the email package has to go MAL> through to maintain such compatibility. That's not a great analogy though. email's got those compatibility modules largely because generators were such a natural fit for three of the methods. I wrote them in Py2.2 first to learn how to use generators and then had to figure out how to backport them. email only works with Python 2.1 and beyond, with a strong bias against the string module. All in all, I'm still not convinced that the compatibility modules are worth it, but for now, I'm sticking with them. They definitely cause pain with distutils. ;/ There's a price to be paid for living on the bleeding edge (or scabbed edge for Python 2.2 ;). But there's also a price to pay for maintaining backwards compatibility and sticking with that forever, More important (to me) than the feature or perceived stability of Py1.5.2, is that so much has been fixed by Python 2.1.3 that it made no sense to use anything earlier. I think/hope that the PBF's Pytie releases will help ease that pain. -Barry
Hi, Skip Montanaro:
mal> BTW, I don't see the tradeoff in saving 11k worth of diskspace for mal> string.py compared to the good vibes we lose in the Python world mal> for this.
It has nothing to do with 11k worth of disk space and everything to do with "there's one (best) way to do it".
In some situations application of zen-rule number 8 (this is "* Although practicality beats purity.") is stronger than zen-rule number 12 (this is "* There should be one-- and preferably only one --obvious way to do it."), which you cited somewhat sloppy above. In my opponion the string module is one such situation and another one is the '<>' operator. Most of my employees work with Modula-2 a lot and we have a huge code base. So they prefer to use '<>' over '!=' in Python also and they will not stop to do so, although the use of '<>' is discouraged in the Python documentation. Regards, Peter -- Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260 office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen, Germany)
skip wrote:
It has nothing to do with 11k worth of disk space and everything to do with "there's one (best) way to do it".
in this case, it sounds more like "purity over practicality" (or maybe "I'm bored, let's get some discussion going") and frankly, it doesn't sound much like you either. didn't you use to be a rather practical programmer? </F>
>> It has nothing to do with 11k worth of disk space and everything to >> do with "there's one (best) way to do it". Fredrik> in this case, it sounds more like "purity over practicality" Fredrik> (or maybe "I'm bored, let's get some discussion going") Fredrik> and frankly, it doesn't sound much like you either. didn't you Fredrik> use to be a rather practical programmer? I am a rather practical programmer, however my programming goals are generally different than yours. I don't write software that is widely distributed. I understand there is bound to be pushback on such a topic. I was also not motivated by boredom. I was motivated by reading this recent addition to the Style Guide: - Avoid the use of the string module; instead use string methods. These are always much faster and share the same API with unicode strings. Now, granted, Guido and Barry weren't saying, "the string module is deprecated". In my original message I even suggested this change in wording to the above: Avoid the use of the string module unless backward-compatibility with versions earlier than Python 2.0 is important; instead use string methods. These are always much faster and share the same API with unicode strings. Still, it sounds to me like for most purposes there is ample reason to advocate not using the string module. Then, my natural inclination is to wonder if it should simply be deprecated. Three functions in the string module are already officially deprecated (atof, atoi, atol since 2.0 - shouldn't their continued use be generating warnings by now?). At least one other should be (capwords and possibly zfill). I would argue that if Guido feels strongly enough to make an "avoid the string module" statement in the Style Guide, a similarly worded statement should also be made at the top of the string module docs. Skip
"SM" == Skip Montanaro <skip@pobox.com> writes:
SM> Now, granted, Guido and Barry weren't saying, "the string SM> module is deprecated". In my original message I even SM> suggested this change in wording to the above: How's this: - Use string methods instead of the string module unless backward-compatibility with versions earlier than Python 2.0 is important. String methods are always much faster and share the same API with unicode strings. -Barry
Skip Montanaro wrote:
Still, it sounds to me like for most purposes there is ample reason to advocate not using the string module.
Sure. Move the documentation for the module into an appendix and add a note saying people should use string methods instead.
Then, my natural inclination is to wonder if it should simply be deprecated.
Removing it will break lots of code. Leaving it alone until Python 3k costs us almost nothing. Can we please just leave it alone? Neil
Removing it will break lots of code. Leaving it alone until Python 3k costs us almost nothing. Can we please just leave it alone?
That's exactly what the PendingDeprecationWarning is for. --Guido van Rossum (home page: http://www.python.org/~guido/)
neil wrote:
Removing it will break lots of code. Leaving it alone until Python 3k costs us almost nothing. Can we please just leave it alone?
fwiw, I grepped a portion of the pythonware.com source code repository. a rough estimate says that we'll have to change 25,000 to 30,000 lines of code that use the string module in one way or another. </F>
Peter> Python 1.5.2 is still very important and even the latest Red Hat Peter> distributions still use it. I think the RedHat/1.5.2 thing is (or should soon be) a red herring. In my message I suggested a two-year/four-release deprecation schedule. That could easily be adjusted depending on what RH decides to do: Research Triangle Park, NC (AP) - RedHat Software announced today that it plans to release RedHat 7.77 during the first week of October 2018. Significant new end user software shipped with this release will include XFree86 17.1.7 and gcc 5.2. Python 1.5.2 has proven amazingly stable however, and too many people rely on it, so it will not be phased out in favor of the current version of Python, 3.15, which was released in August of 2016. Sources close to PythonLabs told the Associated Press that after much discussion on the python-dev mailing list, Guido van Rossum, the creator of the Python programming language, has decided the Python 4.0 interpreter will ship with an automatic 1.5.2 compatibility mode which will be automatically activated when a .pyc file with the proper magic number is loaded. There is no firm word yet when 4.0 is expected to ship. Interested parties are urged to keep their eye on PEP 4327: "Python 4.0 Release Schedule" for information about the planned dates for initial alpha testing. Tests posted to comp.lang.python by Tim Peters suggest that on a large corpus of 20-line floating point and long integer math test scripts rigorously developed over the past 15 years performance was not significantly affected. Sources at Zope Corporation could not be reached for comment in time for this story to confirm any of this information. :-) Skip
On Wed, May 29, 2002, Skip Montanaro wrote:
Peter> Python 1.5.2 is still very important and even the latest Red Hat Peter> distributions still use it.
I think the RedHat/1.5.2 thing is (or should soon be) a red herring. In my message I suggested a two-year/four-release deprecation schedule. That could easily be adjusted depending on what RH decides to do:
Research Triangle Park, NC (AP) - RedHat Software announced today that it plans to release RedHat 7.77 during the first week of October 2018. Significant new end user software shipped with this release will include XFree86 17.1.7 and gcc 5.2. Python 1.5.2 has proven amazingly stable however, and too many people rely on it, so it will not be phased out in favor of the current version of Python, 3.15, which was released in August of 2016.
This is ridiculous. Think about this: how long after ANSI C was released did Python continue to support K&R? That should give you a feeling for long it should take to continue to support old releases for core features. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "In the end, outside of spy agencies, people are far too trusting and willing to help." --Ira Winkler
I'm absolutely against depreceating the string module!
Maybe we need a new concept, "silent deprecation"? We would do we could to discourage the use of modules (like types or string) in the documentation, maybe even removing their documentation altogether, but the interpreter would not issue a warning message. If done well, this could have approximately the same effect as a deprecation warning, but without the negative PR effects. Or maybe we could introduce a new warning category, SilentDeprecationWarning, which is normally ignored (like OverflowWarning already is). This could be turned on explicitly with a -W option so someone wanting to check that their code is future-proof would have an easy way to do so. I do not want to ignore all DeprecationWarning messages by default, as it would defeat the main purpose of the warning. --Guido van Rossum (home page: http://www.python.org/~guido/)
On Wed, May 29, 2002, Guido van Rossum wrote:
Or maybe we could introduce a new warning category, SilentDeprecationWarning, which is normally ignored (like OverflowWarning already is). This could be turned on explicitly with a -W option so someone wanting to check that their code is future-proof would have an easy way to do so.
I do not want to ignore all DeprecationWarning messages by default, as it would defeat the main purpose of the warning.
+1 -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "In the end, outside of spy agencies, people are far too trusting and willing to help." --Ira Winkler
On Wednesday 29 May 2002 03:58 pm, Guido van Rossum wrote:
I'm absolutely against depreceating the string module! ... Or maybe we could introduce a new warning category, SilentDeprecationWarning, which is normally ignored (like OverflowWarning already is). This could be turned on explicitly with a -W option so someone wanting to check that their code is future-proof would have an easy way to do so.
I do not want to ignore all DeprecationWarning messages by default, as it would defeat the main purpose of the warning.
This answers ALL the negatives I had just made in my pessimistic last message and gives me hope in the future again...! *YES!!!* Let's have "silent" deprecation warnings, disabled by default, for features that are _eventually_ going to go away "but not just yet", at the same time as their goner status gets in the docs and for at least (whatever number of releases or duration of time) before moving to "real" (noisy) deprecation. Features might be instantly put in "real" deprecation mode like today, when warranted, but a new and even more gradual option would be allowed for features whose eventual removal needs to be handled ever so slowly and gradually. Super! Why, it might even let <> go away some day... I think it should be ensured that (e.g.) a make test with -W gives no warnings -- basically, that the standard library itself uses no features that are deprecated even silently (though it may of course _provide_ some). Otherwise, users would be discouraged from using -W if that could given them warnings "from the library itself" that are not easily removed by working just on their own code. This needs more thought if we also want the silently-deprecated features to keep being tested by the standard test suite at least optionally. Alex
"GvR" == Guido van Rossum <guido@python.org> writes:
GvR> Or maybe we could introduce a new warning category, GvR> SilentDeprecationWarning, which is normally ignored (like GvR> OverflowWarning already is). This could be turned on GvR> explicitly with a -W option so someone wanting to check that GvR> their code is future-proof would have an easy way to do so. +1 -Barry
GvR> Or maybe we could introduce a new warning category, GvR> SilentDeprecationWarning, which is normally ignored (like GvR> OverflowWarning already is). This could be turned on GvR> explicitly with a -W option so someone wanting to check that GvR> their code is future-proof would have an easy way to do so. BAW> +1 Thank you. Though it is painful to bring up some of these topics (I think I've caught more than my fair share of arrows recently), I find there's often a new way to look at things that makes the problem more tractable. I suggest a migration path from SilentDeprecationWarning to DeprecationWarning. Skip
Guido van Rossum wrote:
Maybe we need a new concept, "silent deprecation"? We would do we could to discourage the use of modules (like types or string) in the documentation, maybe even removing their documentation altogether, but the interpreter would not issue a warning message.
I think explicit is better than implicit. The docs should state that the feature is in the process of being deprecated. First quietly, then loudly, then removed. I would also like to see approximate dates for the schedule. Otherwise, I fear people will say, I saw the message for 2 years, I figured it was safe. The message could be something like: XXX is deprecated and will issue a warning after D/D/D. XXX is deprecated and will be removed after D/D/D. The date could be a version too. Doesn't much matter to me. This approach gives more insight into the general direction of the language also, so I think it's a very good idea (+1). Neal
From: "Guido van Rossum" <guido@python.org>
Maybe we need a new concept, "silent deprecation"? We would do we could to discourage the use of modules (like types or string) in the documentation, maybe even removing their documentation altogether, but the interpreter would not issue a warning message.
+100 Last month, I took the bait and went to comp.lang.py with a proposal to deprecate some builtins. I learned that every builtin (except for input()) has a lot of friends. With each set of flames, I mean comments, I modified my proposal until it evolved to "psuedo-deprecation". Amazingly, this attracted no flames at all. The net result is that psuedo-deprecation seems to be acceptable as way of removing cruft and lowering the sum total pool of working knowledge that a Python programmer has to keep in mind. Instead of de-documenting obsolete tools, I propose moving their documentation to a separate section of the docs set aside for cruft. This makes it possible to understand exactly what old code was doing. It makes sure that the newly dedocumented features don't take on never-documented assumptions. Putting it in a separate place allows us to document the rationale for psuedo-deprecation and to suggest alternatives. Collecting it in one place provides authors with a checklist of things to take out of their book updates. On the warning side of the equation, I have two ideas. While there would not be warnings kicking out automatically, we could add an option or tool that developers could use to temporarily flag use of psuedo-deprecated features. The second idea, is to move responsibility for obsolesence warnings to PyChecker. Raymond Hettinger
[Changed subject] [Raymond Hettinger]
Last month, I took the bait and went to comp.lang.py with a proposal to deprecate some builtins. I learned that every builtin (except for input()) has a lot of friends.
And input() has one important friend. :-)
With each set of flames, I mean comments, I modified my proposal until it evolved to "psuedo-deprecation". Amazingly, this attracted no flames at all. The net result is that psuedo-deprecation seems to be acceptable as way of removing cruft and lowering the sum total pool of working knowledge that a Python programmer has to keep in mind.
I guess this means that people don't mind us telling them which features are obsolete, but they hate it with a vengeance when their programs that they have deployed in the field with end users start spewing warning messages.
Instead of de-documenting obsolete tools, I propose moving their documentation to a separate section of the docs set aside for cruft. This makes it possible to understand exactly what old code was doing.
Hm, getting the docs for a previous version is easy enough. We never throw anything away.
It makes sure that the newly dedocumented features don't take on never-documented assumptions. Putting it in a separate place allows us to document the rationale for psuedo-deprecation and to suggest alternatives. Collecting it in one place provides authors with a checklist of things to take out of their book updates.
I think we can do two things. Either we can keep the docs for deprecated features but clearly mark the chapter or section with a deprecation note at the start, in bold. Or we can remove the docs for deprecated features, and add a chapter "deprecated features" which mentions each deprecated feature without describing it in detail, and provides information about what to use instead.
On the warning side of the equation, I have two ideas. While there would not be warnings kicking out automatically, we could add an option or tool that developers could use to temporarily flag use of psuedo-deprecated features.
This is what I meant by having a SilentDeprecationWarning category -- a special form of the -W option can cause it to spew messages, but by default it won't.
The second idea, is to move responsibility for obsolesence warnings to PyChecker.
This is not always feasible; some things can only be discovered at run-time (e.g. the oct/hex/<< warnings). --Guido van Rossum (home page: http://www.python.org/~guido/)
Note: there's an patch available at http://python.org/sf/561928 Guido van Rossum wrote:
I think we can do two things. Either we can keep the docs for deprecated features but clearly mark the chapter or section with a deprecation note at the start, in bold.
Or we can remove the docs for deprecated features, and add a chapter "deprecated features" which mentions each deprecated feature without describing it in detail, and provides information about what to use instead.
Either of these approaches works for me.
The second idea, is to move responsibility for obsolesence warnings to PyChecker.
This is not always feasible; some things can only be discovered at run-time (e.g. the oct/hex/<< warnings).
Agreed. PyChecker will probably just be an early warning system. Especially since I'm more aggressive on my definition of 'deprecated.' :-) Neal
Note: there's an patch available at http://python.org/sf/561928
Thanks! I agree with the name change to PendingDeprecationWarning. Check it in before I change my mind! :-)
Guido van Rossum wrote:
I think we can do two things. Either we can keep the docs for deprecated features but clearly mark the chapter or section with a deprecation note at the start, in bold.
Or we can remove the docs for deprecated features, and add a chapter "deprecated features" which mentions each deprecated feature without describing it in detail, and provides information about what to use instead.
Either of these approaches works for me.
I personally like the second version better -- by not providing full documentation, it discourages new users more strongly from using deprecated features. --Guido van Rossum (home page: http://www.python.org/~guido/)
From: "Guido van Rossum" <guido@python.org>
Thanks! I agree with the name change to PendingDeprecationWarning. Check it in before I change my mind! :-)
Awesome! Now, down to business. What shall we Silently Deprecate? string module types module (after providing substitutes) apply() oct() hex() Raymond Hettinger
What shall we Silently Deprecate?
string module
Yes, after providing substitutes.
types module (after providing substitutes)
Yes.
apply()
Yes.
oct() hex()
Why? I use these a lot... --Guido van Rossum (home page: http://www.python.org/~guido/)
From: "Guido van Rossum" <guido@python.org>
What shall we Silently Deprecate?
string module
Yes, after providing substitutes.
I commented on the substitutes patch, www.python.org/sf/561832, but thought it would be better to kick the idea around here on Py-dev. Instead of making direct substitutes for the character lists, I propose we take advantage of the opportunity and provide them as mappings rather than strings. That way, we can get O(1) behavior instead of O(n) behavior for code like: if c in str.printable: c='*'. If someone needs to know the contents, they can run str.printable.keys(). Also, because the dictionary is mutable, someone can (at runtime) expand or contract the definitions: str.whitespace.append('_'). Raymond Hettinger
Instead of making direct substitutes for the character lists, I propose we take advantage of the opportunity and provide them as mappings rather than strings. That way, we can get O(1) behavior instead of O(n) behavior for code like: if c in str.printable: c='*'. If someone needs to know the contents, they can run str.printable.keys(). Also, because the dictionary is mutable, someone can (at runtime) expand or contract the definitions: str.whitespace.append('_').
I don't like the idea of making this mutable at all. But perhaps these should be replaced by predicates on strings? Most of the constants already have a predicate companion: whitespace -- isspace() lowercase -- islower() uppercase -- isupper() letters -- isalpha() digits -- isdigit() That leaves: hexdigits -- isxdigit() octdigits -- isodigit() punctuation -- ispunct() printable -- isprint() Perhaps we should add isalnum, iscntrl, is graph, to match <ctype.h>? Or perhaps not? (Maybe I'd also like to add isword(), which would be isalnum() or '_' -- this is the definition of \w in the re module.) --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
Most of the constants already have a predicate companion:
whitespace -- isspace() lowercase -- islower() uppercase -- isupper() letters -- isalpha() digits -- isdigit()
That leaves:
hexdigits -- isxdigit() octdigits -- isodigit() punctuation -- ispunct() printable -- isprint()
Perhaps we should add isalnum, iscntrl, is graph, to match <ctype.h>? Or perhaps not? (Maybe I'd also like to add isword(), which would be isalnum() or '_' -- this is the definition of \w in the re module.)
FYI, Unicode has these: {"islower", (PyCFunction) unicode_islower, METH_NOARGS, islower__doc__}, {"isupper", (PyCFunction) unicode_isupper, METH_NOARGS, isupper__doc__}, {"istitle", (PyCFunction) unicode_istitle, METH_NOARGS, istitle__doc__}, {"isspace", (PyCFunction) unicode_isspace, METH_NOARGS, isspace__doc__}, {"isdecimal", (PyCFunction) unicode_isdecimal, METH_NOARGS, isdecimal__doc__}, {"isdigit", (PyCFunction) unicode_isdigit, METH_NOARGS, isdigit__doc__}, {"isnumeric", (PyCFunction) unicode_isnumeric, METH_NOARGS, isnumeric__doc__}, {"isalpha", (PyCFunction) unicode_isalpha, METH_NOARGS, isalpha__doc__}, {"isalnum", (PyCFunction) unicode_isalnum, METH_NOARGS, isalnum__doc__}, Note that unlike for ASCII strings, the characters all have defined categories. Strings miss a few of these: {"islower", (PyCFunction)string_islower, METH_NOARGS, islower__doc__}, {"isupper", (PyCFunction)string_isupper, METH_NOARGS, isupper__doc__}, {"isspace", (PyCFunction)string_isspace, METH_NOARGS, isspace__doc__}, {"isdigit", (PyCFunction)string_isdigit, METH_NOARGS, isdigit__doc__}, {"istitle", (PyCFunction)string_istitle, METH_NOARGS, istitle__doc__}, {"isalpha", (PyCFunction)string_isalpha, METH_NOARGS, isalpha__doc__}, {"isalnum", (PyCFunction)string_isalnum, METH_NOARGS, isalnum__doc__}, -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/
FYI, Unicode has these:
{"islower", (PyCFunction) unicode_islower, METH_NOARGS, islower__doc__}, {"isupper", (PyCFunction) unicode_isupper, METH_NOARGS, isupper__doc__}, {"istitle", (PyCFunction) unicode_istitle, METH_NOARGS, istitle__doc__}, {"isspace", (PyCFunction) unicode_isspace, METH_NOARGS, isspace__doc__}, {"isdecimal", (PyCFunction) unicode_isdecimal, METH_NOARGS, isdecimal__doc__}, {"isdigit", (PyCFunction) unicode_isdigit, METH_NOARGS, isdigit__doc__}, {"isnumeric", (PyCFunction) unicode_isnumeric, METH_NOARGS, isnumeric__doc__}, {"isalpha", (PyCFunction) unicode_isalpha, METH_NOARGS, isalpha__doc__}, {"isalnum", (PyCFunction) unicode_isalnum, METH_NOARGS, isalnum__doc__},
Note that unlike for ASCII strings, the characters all have defined categories.
Strings miss a few of these:
{"islower", (PyCFunction)string_islower, METH_NOARGS, islower__doc__}, {"isupper", (PyCFunction)string_isupper, METH_NOARGS, isupper__doc__}, {"isspace", (PyCFunction)string_isspace, METH_NOARGS, isspace__doc__}, {"isdigit", (PyCFunction)string_isdigit, METH_NOARGS, isdigit__doc__}, {"istitle", (PyCFunction)string_istitle, METH_NOARGS, istitle__doc__}, {"isalpha", (PyCFunction)string_isalpha, METH_NOARGS, isalpha__doc__}, {"isalnum", (PyCFunction)string_isalnum, METH_NOARGS, isalnum__doc__},
I think all of the unicode methods should be added to 8-bit strings, even if they are just aliases for others (what's title case applied to Latin-1? I suppose same as upper case?). --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
FYI, Unicode has these:
{"islower", (PyCFunction) unicode_islower, METH_NOARGS, islower__doc__}, {"isupper", (PyCFunction) unicode_isupper, METH_NOARGS, isupper__doc__}, {"istitle", (PyCFunction) unicode_istitle, METH_NOARGS, istitle__doc__}, {"isspace", (PyCFunction) unicode_isspace, METH_NOARGS, isspace__doc__}, {"isdecimal", (PyCFunction) unicode_isdecimal, METH_NOARGS, isdecimal__doc__}, {"isdigit", (PyCFunction) unicode_isdigit, METH_NOARGS, isdigit__doc__}, {"isnumeric", (PyCFunction) unicode_isnumeric, METH_NOARGS, isnumeric__doc__}, {"isalpha", (PyCFunction) unicode_isalpha, METH_NOARGS, isalpha__doc__}, {"isalnum", (PyCFunction) unicode_isalnum, METH_NOARGS, isalnum__doc__},
Note that unlike for ASCII strings, the characters all have defined categories.
Strings miss a few of these:
{"islower", (PyCFunction)string_islower, METH_NOARGS, islower__doc__}, {"isupper", (PyCFunction)string_isupper, METH_NOARGS, isupper__doc__}, {"isspace", (PyCFunction)string_isspace, METH_NOARGS, isspace__doc__}, {"isdigit", (PyCFunction)string_isdigit, METH_NOARGS, isdigit__doc__}, {"istitle", (PyCFunction)string_istitle, METH_NOARGS, istitle__doc__}, {"isalpha", (PyCFunction)string_isalpha, METH_NOARGS, isalpha__doc__}, {"isalnum", (PyCFunction)string_isalnum, METH_NOARGS, isalnum__doc__},
I think all of the unicode methods should be added to 8-bit strings, even if they are just aliases for others (what's title case applied to Latin-1? I suppose same as upper case?).
for i in range(256): ... assert unichr(i).istitle() == unichr(i).isupper() ...
Seems so :-) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ Meet us at EuroPython 2002: http://www.europython.org/
On Wednesday 29 May 2002 09:32 pm, Guido van Rossum wrote: ...
{"istitle", (PyCFunction)string_istitle, METH_NOARGS, istitle__doc__}, ... I think all of the unicode methods should be added to 8-bit strings, even if they are just aliases for others (what's title case applied to Latin-1? I suppose same as upper case?).
Current Python strings DO have .istitle() -- it means "the first letter of each substring made of letters is uppercase, all the other letters in the substring are lowercase". (It also means the string DOES have some letters -- I always find it quite tricky that all the string iswhatever methods return False when called on an empty string).
"Pol23Zap And Plee+Kuup!".istitle() True
Alex
Guido van Rossum <guido@python.org> writes:
I think all of the unicode methods should be added to 8-bit strings, even if they are just aliases for others (what's title case applied to Latin-1? I suppose same as upper case?).
So those would be locale dependent? (the Unicode ones are not). Regards, Martin
I think all of the unicode methods should be added to 8-bit strings, even if they are just aliases for others (what's title case applied to Latin-1? I suppose same as upper case?).
So those would be locale dependent? (the Unicode ones are not).
Yes. isalpha() etc. for 8-bit strings are already locale dependent aren't they? BTW, how about deprecating strop while we're at it? --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum <guido@python.org> writes:
I think all of the unicode methods should be added to 8-bit strings, even if they are just aliases for others (what's title case applied to Latin-1? I suppose same as upper case?).
So those would be locale dependent? (the Unicode ones are not).
Yes. isalpha() etc. for 8-bit strings are already locale dependent aren't they?
Yes, certainly. I'm just confirming, since this *is* a notable difference to Unicode strings.
BTW, how about deprecating strop while we're at it?
That seems acceptable to me, once string.py stops importing it. I wonder how much time after deprecating a builtin or extension module we should stop building it automatically in setup.py. Regards, Martin
I think all of the unicode methods should be added to 8-bit strings, even if they are just aliases for others (what's title case applied to Latin-1? I suppose same as upper case?).
So those would be locale dependent? (the Unicode ones are not).
Yes. isalpha() etc. for 8-bit strings are already locale dependent aren't they?
Yes, certainly. I'm just confirming, since this *is* a notable difference to Unicode strings.
Yes, but that's because locale-dependency is not a Unicode concept!
BTW, how about deprecating strop while we're at it?
That seems acceptable to me, once string.py stops importing it.
I wonder how much time after deprecating a builtin or extension module we should stop building it automatically in setup.py.
Me too. There are probably still places like Greg Stein's original import hack that import strop because they haven't been converted to string methods. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
BTW, how about deprecating strop while we're at it?
Right now in string.py, there is: from strop import maketrans, lowercase, uppercase, whitespace letters = lowercase + uppercase However, lowercase, uppercase, and letters are already defined in string.py. It seems the only function being used from strop is maketrans. Is this correct? Should the others be removed from string.py? Neal
Right now in string.py, there is:
from strop import maketrans, lowercase, uppercase, whitespace letters = lowercase + uppercase
However, lowercase, uppercase, and letters are already defined in string.py. It seems the only function being used from strop is maketrans. Is this correct? Should the others be removed from string.py?
strop.letters etc. are fixed when locale.setconv(LC_CTYPE, ...) is used. string.letters etc. are *also* fixed, but if string hasn't been imported yet when the locale is set, string gets the properly localized versions from strop (which does the right thing upon initialization to reflect the locale). This mechanism would have to be copied into string.py somehow. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
strop.letters etc. are fixed when locale.setconv(LC_CTYPE, ...) is used. string.letters etc. are *also* fixed, but if string hasn't been imported yet when the locale is set, string gets the properly localized versions from strop (which does the right thing upon initialization to reflect the locale). This mechanism would have to be copied into string.py somehow.
Michael also schooled me on this. I made a comment on Thomas Heller's patch http://python.org/sf/561832 to indicate this info. His patch adds letters, digits, etc. but not maketrans(). Neal
Michael also schooled me on this. I made a comment on Thomas Heller's patch http://python.org/sf/561832 to indicate this info. His patch adds letters, digits, etc. but not maketrans().
I've rejected that patch -- we should use isxxx() methods instead of exposing the sets as variables. --Guido van Rossum (home page: http://www.python.org/~guido/)
[Guido van Rossum]
Perhaps we should add isalnum, iscntrl, is graph, to match <ctype.h>? Or perhaps not? (Maybe I'd also like to add isword(), which would be isalnum() or '_' -- this is the definition of \w in the re module.)
This reminds me that I often miss, in the standard `ctype.h' and related, a function that would un-combine a character into its base character and its diacritic, and the complementary re-combining function. Even if this might be easier for Latin-1, it is difficult to design something general enough. Characters may have a more complex structure than a mere base and single diacritic. I do not know what to suggest. -- François Pinard http://www.iro.umontreal.ca/~pinard
This reminds me that I often miss, in the standard `ctype.h' and related, a function that would un-combine a character into its base character and its diacritic, and the complementary re-combining function.
Even if this might be easier for Latin-1, it is difficult to design something general enough. Characters may have a more complex structure than a mere base and single diacritic. I do not know what to suggest.
I bet the Unicode standard has a standard way to do this. Maybe we can implement that, and then project the same interface on 8-bit characters? Of course character encoding issues might get in the way if <ctype.h> doesn't provide the data -- so you may be better off doing this in Unicode only. (We must never assume that 8-bit strings contain Latin-1.) --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum <guido@python.org> writes:
This reminds me that I often miss, in the standard `ctype.h' and related, a function that would un-combine a character into its base character and its diacritic, and the complementary re-combining function. [...] I bet the Unicode standard has a standard way to do this.
This is called 'unicode normalization forms'. Each "pre-combined" character can also be represented as a base character, and a "combining diacritic". There are symmetric normalization forms: NFC favours pre-combined characters, NFD favours combining characters. There is also a "compatibility decomposition" (K), where e.g. ANGSTROM SIGN decomposes to LATIN CAPITAL LETTER A WITH RING ABOVE.
Maybe we can implement that, and then project the same interface on 8-bit characters?
Not really. Needing to know the character set is one issue; the other issue is that the stand-alone diacritic characters in ASCII are *not* combining. We could certainly provide a mapping between the Unicode combining diacritics and the stand-alone diacritics, say as a codec, but that would be quite special-purpose. Providing a good normalization library is necessary, though, since many other algorithms (both from W3C and IETF) require Unicode normalization as part of the processing (usually to NFKC). Regards, Martin
François Pinard wrote:
This reminds me that I often miss, in the standard `ctype.h' and related, a function that would un-combine a character into its base character and its diacritic, and the complementary re-combining function.
import unicodedata def uncombine(char): chars = unicodedata.decomposition(unichr(ord(char))).split() if not chars: return [char] return [unichr(int(x, 16)) for x in chars if x[0] != "<"] for char in "François": print uncombine(char) ['F'] ['r'] ['a'] ['n'] [u'c', u'\u0327'] ['o'] ['i'] ['s'] (to go the other way, store all uncombinations longer than one character in a dictionary) </F>
On Wednesday 29 May 2002 07:28 pm, Raymond Hettinger wrote: ...
strings. That way, we can get O(1) behavior instead of O(n) behavior for code like: if c in str.printable: c='*'. If someone needs to know the contents, they can run str.printable.keys(). Also, because the dictionary is mutable, someone can (at runtime) expand or contract the definitions: str.whitespace.append('_').
append would of course not work on a dictionary, but the prospect of allowing easy mutation of fundamental built-ins is quite negative in Python -- goes against the grain of the language. A read-only dictionary might be OK. Alex
strings. That way, we can get O(1) behavior instead of O(n) behavior for code like: if c in str.printable: c='*'. If someone needs to know
From: "Alex Martelli" <aleax@aleax.it> the
contents, they can run str.printable.keys(). Also, because the dictionary is mutable, someone can (at runtime) expand or contract the definitions: str.whitespace.append('_').
append would of course not work on a dictionary, but the prospect of allowing easy mutation of fundamental built-ins is quite negative in Python -- goes against the grain of the language. A read-only dictionary might be OK.
I like the read-only dictionary better than the boolean test methods. It minimizes the effort in upgrading existing code of the form: for c in string.lowercase: do something if c in string.lowercase: do something I can global search/replace string.lowercase with str.lower in dictionary form and everything will run fine (and faster too). Also, I like the mapping because provides a way to see the membership. With the string form or the mapping form, it's easy to find-out exactly what is defined as whitespace. If there is a C coded boolean test, I have to filter the whole alphabet or look it up in the docs. If we can get some agreement that this is the way to go, I can work with the OP on a revised patch so we get the string module silently deprecated. Raymond Hettinger
I like the read-only dictionary better than the boolean test methods. It minimizes the effort in upgrading existing code of the form:
for c in string.lowercase: do something if c in string.lowercase: do something
I can global search/replace string.lowercase with str.lower in dictionary form and everything will run fine (and faster too).
I think this is not enough motivation. We have a whole slew of test methods already. Also the dict approach doesn't scale to Unicode (the dicts would have to be enormous) while the test method approach easily scales to Unicode (it's already implemented that way there).
Also, I like the mapping because provides a way to see the membership. With the string form or the mapping form, it's easy to find-out exactly what is defined as whitespace. If there is a C coded boolean test, I have to filter the whole alphabet or look it up in the docs.
A very minor advantage indeed -- and again one that doesn't scale to Unicode.
If we can get some agreement that this is the way to go, I can work with the OP on a revised patch so we get the string module silently deprecated.
Maybe you can do a patch for isxxx() methods instead? --Guido van Rossum (home page: http://www.python.org/~guido/)
From: "Guido van Rossum" <guido@python.org>
I like the read-only dictionary better than the boolean test methods. It minimizes the effort in upgrading existing code of the form:
for c in string.lowercase: do something if c in string.lowercase: do something
Maybe you can do a patch for isxxx() methods instead?
Will do. Raymond Hettinger
On Wednesday 29 May 2002 06:42 pm, Guido van Rossum wrote: ...
oct() hex()
Why? I use these a lot...
I assume the duplication of oct and hex wrt '%o'% and '%x'% was the reason to suggest silently-deprecating the former (trying to have 'just one obvious way' and all that). Personally, what seems weird to me is that we have a way to _parse_ numbers in any base (thanks to the 2-args form of str) but not one to _emit_ them in any base -- an issue of asymmetry more than anything else (binary is probably the only format besides oct and hex that would see significant use -- as I would guess may be the case for 2-arguments str today). The % operator is probably a bit too obscure / unreadable to consider it a general substitute for oct and hex, alas. Just MHO of course. Alex
[Alex Martelli]
I assume the duplication of oct and hex wrt '%o'% and '%x'% was the reason to suggest silently-deprecating the former (trying to have 'just one obvious way' and all that).
The same kind of reason says that `str()' and `repr()' could be faded out in favour of `%s' and `%r' specifiers. This is not likely to happen! :-) -- François Pinard http://www.iro.umontreal.ca/~pinard
On Wednesday 29 May 2002 07:39 pm, François Pinard wrote:
[Alex Martelli]
I assume the duplication of oct and hex wrt '%o'% and '%x'% was the reason to suggest silently-deprecating the former (trying to have 'just one obvious way' and all that).
The same kind of reason says that `str()' and `repr()' could be faded out in favour of `%s' and `%r' specifiers. This is not likely to happen! :-)
Having both repr AND the peculiar `...` operator is another one of those things which I find _really_ hard to explain without handwaving and without sounding negative about Python, a language about which I'm actually enthusiastic. I'd rather lose the `...` (via pendingsilentdeprecation or whatever). As to repr itself, I'd be +0 on taking it out of the builtins, but that's hardly a major issue, nor of course a realistic prospect. str is obviously quite a different kettle of fish -- it's a TYPE and thus it cannot be substituted by whatever operator. It would also be the natural way to put a format-with-whatever-number-base functionality (2-arg str, just like parsing with arbitrary base is 2-arg int -- perfect!). Alex
[Alex Martelli]
On Wednesday 29 May 2002 06:42 pm, Guido van Rossum wrote: ...
oct() hex()
Why? I use these a lot...
I assume the duplication of oct and hex wrt '%o'% and '%x'% was the reason to suggest silently-deprecating the former (trying to have 'just one obvious way' and all that).
Hi, people. I'm revising many accumulated notes, while writing the draft of a Python style and migration guide (in French) for a small team of Python programmers, here. By the way, I thank you all for the richness of the exchanged ideas in that area, lately. Also, poking around, I see even a bit deeper than before how beautiful the Python project is! Stumbling on the above message, I feel like making a further comment. When I was learning Python, I found elegant to discover that Python had all that is required so one could rewrite the `FORMAT % THINGS' operator, if one wanted to. If we deprecate built-ins (like `repr', `hex' and `oct') in favour of leaving `%' as the only way, we would loose that elegance. Moreover, it might be more speedy not having to go through the interpretation of a format string, and this might matter in some circumstances. -- François Pinard http://www.iro.umontreal.ca/~pinard
Raymond> What shall we Silently Deprecate? I see a slogan for the next Python Conference: Nobody expects the Silent Deprecation :-) Skip
On Wed, May 29, 2002 at 12:40:02PM -0400, Raymond Hettinger wrote:
From: "Guido van Rossum" <guido@python.org>
Thanks! I agree with the name change to PendingDeprecationWarning. Check it in before I change my mind! :-)
Awesome! Now, down to business. What shall we Silently Deprecate?
string module types module (after providing substitutes)
It looks like many of the names in the types module already have substitutes in __builtins__:
import types for name, t in types.__dict__.items(): ... if type(t) is type and hasattr(__builtins__, t.__name__): ... print 'types.%s -> %s' % (name, t.__name__) ... types.IntType -> int types.TypeType -> type types.StringType -> str types.FloatType -> float types.ObjectType -> object types.DictionaryType -> dict types.FileType -> file types.DictType -> dict types.ListType -> list types.TupleType -> tuple types.LongType -> long types.BufferType -> buffer types.UnicodeType -> unicode types.ComplexType -> complex types.SliceType -> slice types.XRangeType -> xrange
Some of these are new in 2.2 (like object, dict and file). Some of them used to be functions before Python 2.2 (like str, int and list). Three of them are still builtin functions in Python 2.2: xrange, buffer and slice. Perhaps they should also be converted to types for consistency. Some more factory functions that could be unified with the type of the objects they create module can be found in the new module. They, too can be used as substitutes for names in the types module.
import new for name, t in types.__dict__.items(): ... if type(t) is type and hasattr(new, t.__name__): ... print 'types.%s -> new.%s -> %s' % (name, t.__name__, t.__name__) types.CodeType -> new.code -> code types.ModuleType -> new.module -> module types.LambdaType -> new.function -> function types.InstanceType -> new.instance -> instance types.FunctionType -> new.function -> function
There are two that almost made it to this list but the name of the factory function in module new is not exactly the same as the type's __name__: types.MethodType -> new.instancemethod -> instancemethod ('instance method') types.ClassType -> new.classobj -> classobj ('class') For instancemethod it's easy to remove the space from the type's __name__. The word class is a reserved word. The type's __name__ could be changed to 'classobj' to match the factory function's name. Some other alternatives I can think of are 'class_', 'ClassType' or 'classtype'. Instances of the remaining types can't be instanciated from Python code. Most of them can be safely identified by their __name__: types.BuiltinMethodType -> builtin_function_or_method types.BuiltinFunctionType -> builtin_function_or_method types.TracebackType -> traceback types.GeneratorType -> generator types.FrameType -> frame types.NoneType -> NoneType The last two remaining troublemakers: types.DictProxyType -> dict_proxy ('dict-proxy') 'dict-proxy' is not a valid Python identifier. types.EllipsisType -> EllipsisType ('ellipsis') The name 'ellipsis' is easily confused with 'Ellipsis'. The name EllipsisType is consistent with NoneType. Both are the type of a specific singleton object. Should all these names be added to __builtins__? Many of them are not very useful in everyday programming. Perhaps the less important ones can be put away in some module. Now how should that module be named? Ummm... maybe 'types'? :-) Oren
It looks like many of the names in the types module already have substitutes in __builtins__:
This was all covered a few days ago. :-)
Some of these are new in 2.2 (like object, dict and file). Some of them used to be functions before Python 2.2 (like str, int and list). Three of them are still builtin functions in Python 2.2: xrange, buffer and slice. Perhaps they should also be converted to types for consistency.
You can help by contributing patches for these three. (Let us know if you plan to do this, so others can relax.)
Some more factory functions that could be unified with the type of the objects they create module can be found in the new module. They, too can be used as substitutes for names in the types module.
import new for name, t in types.__dict__.items(): ... if type(t) is type and hasattr(new, t.__name__): ... print 'types.%s -> new.%s -> %s' % (name, t.__name__, t.__name__) types.CodeType -> new.code -> code types.ModuleType -> new.module -> module types.LambdaType -> new.function -> function types.InstanceType -> new.instance -> instance types.FunctionType -> new.function -> function
Except that the new module has the wrong name. But making all these types proper factories would be a first step that's useful anyway. Patch please?
There are two that almost made it to this list but the name of the factory function in module new is not exactly the same as the type's __name__:
types.MethodType -> new.instancemethod -> instancemethod ('instance method')
So change the type's __name__.
types.ClassType -> new.classobj -> classobj ('class')
Hopefully the classic class will disappear at the same time as types.py is removed (in 3.0).
For instancemethod it's easy to remove the space from the type's __name__. The word class is a reserved word. The type's __name__ could be changed to 'classobj' to match the factory function's name. Some other alternatives I can think of are 'class_', 'ClassType' or 'classtype'.
Or 'classic_class'.
Now how should that module be named? Ummm... maybe 'types'? :-)
This may be the best solution after all. --Guido van Rossum (home page: http://www.python.org/~guido/)
On Thu, May 30, 2002 at 08:35:02AM -0400, Guido van Rossum wrote:
Some of these are new in 2.2 (like object, dict and file). Some of them used to be functions before Python 2.2 (like str, int and list). Three of them are still builtin functions in Python 2.2: xrange, buffer and slice. Perhaps they should also be converted to types for consistency.
You can help by contributing patches for these three. (Let us know if you plan to do this, so others can relax.)
Okay. It will probably take me 20 time longer than someone who knows his way better around the CPython sources, but I guest I gotta start somewhere. Oren
This is not always feasible; some things can only be discovered at run-time (e.g. the oct/hex/<< warnings).
Agreed. PyChecker will probably just be an early warning system. Especially since I'm more aggressive on my definition of 'deprecated.' :-)
Sometimes I wish pychecker was integrated with the distro... But I'm sure there are good reasons for avoiding this. -- Mike -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html
Sometimes I wish pychecker was integrated with the distro... But I'm sure there are good reasons for avoiding this.
Mostly that the PyChecker development cycle is several orders of magnitude faster than Python's, so at best you would have an out-of-date version of PyChecker in the distro. Once PyChecker slows down a bit, I'd be happy to incorporate it. --Guido van Rossum (home page: http://www.python.org/~guido/)
Title changed. Guido van Rossum wrote:
Sometimes I wish pychecker was integrated with the distro... But I'm sure there are good reasons for avoiding this.
Mostly that the PyChecker development cycle is several orders of magnitude faster than Python's, so at best you would have an out-of-date version of PyChecker in the distro. Once PyChecker slows down a bit, I'd be happy to incorporate it.
Pychecker development has slowed down a bit, but there are some other reasons to not include it just yet. There's several issues that are slowly converging. The new compiler which Jeremy did a bunch of work on is an important part. Ideally, pychecker would be implemented on top of the new compiler. This will make integration of pychecker easier and should also allow it to work with jython. There's also been work on using ASTs for pychecker, rather than byte codes. These are 2 important parts which are on-going. I've thought about returning to the compiler, but I'm also keen to clean up a lot of cruft. I think that's more important. Generally, I'd like to finish cleaning up the deprecation issues, which include: using PendingDeprecations, Py_DEPRECATED in C headers, and fixing the docs. I'm still thinking about optimizations too (remove SET_LINENO and others). And finally, the compiler and pychecker. (Also, generally improving test coverage is hrown in somewhere.) I'm hopeful that 2.4 may be a good time to get the compiler and pychecker into Python. But I suspect that pychecker will still be maintained separately for a while. Neal
Skip Montanaro <skip@pobox.com> writes:
How about deprecating the string module? In the past, this has presented two major problems. First, its near ubiquity in Python programs before the availability of string methods. Second, the presence of a few useful data objects (digits, uppercase, etc). The first problem can be solved by extending the deprecation time suitably (two years/four releases?) I think the second problem can be solved by adding those data objects to either sys or locale(preferably the latter).
I think one needs to offer the alternatives first, and deprecate the module then. So have alternatives for *everything* in string available in 2.3, deprecate it in 2.4, add a DeprecationWarning in 2.5. Regards, Martin
>> How about deprecating the string module? Martin> I think one needs to offer the alternatives first, and deprecate Martin> the module then. So have alternatives for *everything* in string Martin> available in 2.3, deprecate it in 2.4, add a DeprecationWarning Martin> in 2.5. I don't see that as being a major problem. As I mentioned in my reply to Peter Funk, I believe you can probably let the deprecation schedule slip according to how important continued 1.5.2 compliance is determined to be down the road. That is, deprecate it no earlier than 2.4 and add a DeprecationWarning no sooner than one or two releases after that. The following string module functions don't seem to have equivalent string methods: zfill - only because string.zfill accepts non-string arguments, otherwise it is already available capwords - I believe it was explicitly decided to not implement this as a string method maketrans - shouldn't be too hard to rewrite in C as a method of str. Also, the data objects obviously have to be put somewhere. Before doing any of this, we need to decide where this stuff belongs. I doubt the technical effort involved will be too challenging. Skip
Skip Montanaro <skip@pobox.com> writes:
Also, the data objects obviously have to be put somewhere. Before doing any of this, we need to decide where this stuff belongs. I doubt the technical effort involved will be too challenging.
So we don't need to make a decision today, right? We can delay that until those easy problems are solved. Regards, Martin
Also, the data objects obviously have to be put somewhere. Before doing any of this, we need to decide where this stuff belongs. I doubt the technical effort involved will be too challenging.
So we don't need to make a decision today, right? We can delay that until those easy problems are solved.
What about my proposal to get rid of the data objects and add a few isxxx() methods instead? I don't think that the data objects have any other function except for membership testing, and a method can be faster. The method approach will also make locale-awareness easier (since the method can use <ctype.h> which is already locale-aware). --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum <guido@python.org> writes:
Also, the data objects obviously have to be put somewhere. Before doing any of this, we need to decide where this stuff belongs. I doubt the technical effort involved will be too challenging.
So we don't need to make a decision today, right? We can delay that until those easy problems are solved.
What about my proposal to get rid of the data objects and add a few isxxx() methods instead?
That's a good thing to do, I agree.
I don't think that the data objects have any other function except for membership testing, and a method can be faster.
That is not true. Asking google for "string.letters" brings, as the first hit, non_letters = string.translate(norm, norm, string.letters) I also found random.choice(string.lowercase[:26] + string.uppercase[:26] + string.digits) This is, of course, nonsense, since it tries to "unlocalize" string.lowercase, where a string literal would have been better. I'm sure people have found other uses for these constants. Regards, Martin
I don't think that the data objects have any other function except for membership testing, and a method can be faster.
That is not true. Asking google for "string.letters" brings, as the first hit,
non_letters = string.translate(norm, norm, string.letters)
I also found
random.choice(string.lowercase[:26] + string.uppercase[:26] + string.digits)
This is, of course, nonsense, since it tries to "unlocalize" string.lowercase, where a string literal would have been better.
I'm sure people have found other uses for these constants.
That just shows we have to be very slow before we remove the string module. I think "not until 3.0" is slow enough. These "use cases" don't convince me that there's a legitimate use case for string.letters etc. that the methods don't cover. --Guido van Rossum (home page: http://www.python.org/~guido/)
From: "Guido van Rossum" <guido@python.org>
These "use cases" don't convince me that there's a legitimate use case for string.letters etc. that the methods don't cover.
This is funny. In the C++ community there's a nearly unanimous consensus that way too much of the functionality of the standard strings is expressed as member functions. -Dave
These "use cases" don't convince me that there's a legitimate use case for string.letters etc. that the methods don't cover.
This is funny. In the C++ community there's a nearly unanimous consensus that way too much of the functionality of the standard strings is expressed as member functions.
Interesting. Python used to have the same attitude, hence the string module -- but the existence of multiple string types made methods more attractive. What's the alternative proposed for C++? --Guido van Rossum (home page: http://www.python.org/~guido/)
From: "Guido van Rossum" <guido@python.org>
These "use cases" don't convince me that there's a legitimate use case for string.letters etc. that the methods don't cover.
This is funny. In the C++ community there's a nearly unanimous consensus that way too much of the functionality of the standard strings is expressed as member functions.
Interesting. Python used to have the same attitude, hence the string module -- but the existence of multiple string types made methods more attractive.
What's the alternative proposed for C++?
Free functions at namespace scope. The analogy would be module-level functions in Python. C++ also has multiple string types, but the availability of overloading makes this approach practical (is it time for Python multimethods yet?) If I were to make arguments against string member functions in Python I'd be talking about the degree of coupling between algorithms and data structures, how it interferes with genericity, and the difficulty that users will have in making "string-like" types... but-i-would-never-make-such-silly-arguments-ly y'rs, dave
david wrote:
What's the alternative proposed for C++?
Free functions at namespace scope. The analogy would be module-level functions in Python. C++ also has multiple string types, but the availability of overloading makes this approach practical
it works pretty well in Python too, of course: the string module delegates to object methods for all functions; e.g. "string.join()" uses the "join" implementation method to do the actual work, just like "len()" uses "__len__" (etc). </F>
I think the "nearly unanimous" is a bit overstated. Some people like the older C-style free functions, but I think this is just old habits fighting on until the bitter end. My (perhaps wacko) preference is to carefully segregate Strings and CharacterSets. After all, iswhitespace() maps a fact about a character set to a specific string instance. I think that the classic ASCII, the various ISO 8859 variants, and the Unicode character sets should be objectified and their attributes include strings of whitespace, uppercase, lowercase, etc., etc. --- David Abrahams <david.abrahams@rcn.com> wrote:
From: "Guido van Rossum" <guido@python.org>
These "use cases" don't convince me that there's a legitimate use case for string.letters etc. that the methods don't cover.
This is funny. In the C++ community there's a nearly unanimous consensus that way too much of the functionality of the standard strings is expressed as member functions.
Interesting. Python used to have the same attitude, hence the string module -- but the existence of multiple string types made methods more attractive.
What's the alternative proposed for C++?
Free functions at namespace scope. The analogy would be module-level functions in Python. C++ also has multiple string types, but the availability of overloading makes this approach practical (is it time for Python multimethods yet?)
If I were to make arguments against string member functions in Python I'd be talking about the degree of coupling between algorithms and data structures, how it interferes with genericity, and the difficulty that users will have in making "string-like" types...
but-i-would-never-make-such-silly-arguments-ly y'rs, dave
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev
===== -- S. Lott, CCP :-{) S_LOTT@YAHOO.COM http://www.mindspring.com/~slott1 Buccaneer #468: KaDiMa Macintosh user: drinking upstream from the herd. __________________________________________________ Do You Yahoo!? Yahoo! - Official partner of 2002 FIFA World Cup http://fifaworldcup.yahoo.com
From: "Steven Lott" <s_lott@yahoo.com>
I think the "nearly unanimous" is a bit overstated.
Might be.
Some people like the older C-style free functions, but I think this is just old habits fighting on until the bitter end.
It's not just that. People are generally unhappy with "kitchen sink" classes (admittedly C++ strings have a horrible case of this because of the combination of STL and legacy interfaces). Also, member functions tend to be difficult for generic programs.
My (perhaps wacko) preference is to carefully segregate Strings and CharacterSets.
After all, iswhitespace() maps a fact about a character set to a specific string instance.
I think that the classic ASCII, the various ISO 8859 variants, and the Unicode character sets should be objectified and their attributes include strings of whitespace, uppercase, lowercase, etc., etc.
I think we're way OT here, but do you know about char_traits? -Dave
It's not just that. People are generally unhappy with "kitchen sink" classes (admittedly C++ strings have a horrible case of this because of the combination of STL and legacy interfaces). Also, member functions tend to be difficult for generic programs.
Is this still relevant to Python? Why are C++ member functions difficult to generic programs? Does the same apply to Python methods? --Guido van Rossum (home page: http://www.python.org/~guido/)
From: "Guido van Rossum" <guido@python.org>
Is this still relevant to Python?
Possibly.
Why are C++ member functions difficult to generic programs?
1. built-in types don't have member functions, so there's no way to get them to work like something that does. For that reason, when a generic function relies on member functions, it's "not very generic". 2. By the same token, if a generic function requires that a type have particular member functions, some class types may be ineligible for use with that function. If the requirement is simply that the appropriate free functions exist, it's usually possible to write them. This becomes a factor when using generic library A with a type from library B.
Does the same apply to Python methods?
#1 obviously doesn't, except that of course the built-in types have a fixed set of methods. #2 might apply. The general trend towards decoupling algorithms and data structures in C++ is one of those Good Things (tm). -Dave
Guido van Rossum <guido@python.org> writes:
Is this still relevant to Python? Why are C++ member functions difficult to generic programs? Does the same apply to Python methods?
In a generic algorithm foo, you can write def foo1(x): bar(x) if you have global functions. With methods, you need to write def foo1(x): x.bar() which means that bar must be a method of x. This might be difficult to achieve if x is of a class that you cannot control. In C++, it is then still possible to define a function def bar(x of-type-X): pass which, by means of overload resolution, will be found from foo1 automatically. In Python, this is not so easy since you don't have overloading. Regards, Martin
In python, you don't need overloading, you have a variety of optional parameter mechanisms. I think the "member functions" issues from C++ don't apply to Python becuase C++ is strongly typed, meaning that many similar functions have to be written with slightly different type signatures. The lack of strong typing makes it practical to write generic operations. I find that use of free functions defeats good object-oriented design and leads to functionality being informally bound by a cluster of free functions that have similar names. I'm suspicious of this, finding it tiresome to maintain and debug. --- "Martin v. Loewis" <martin@v.loewis.de> wrote:
Guido van Rossum <guido@python.org> writes:
Is this still relevant to Python? Why are C++ member functions difficult to generic programs? Does the same apply to Python methods?
In a generic algorithm foo, you can write
def foo1(x): bar(x)
if you have global functions. With methods, you need to write
def foo1(x): x.bar()
which means that bar must be a method of x. This might be difficult to achieve if x is of a class that you cannot control. In C++, it is then still possible to define a function
def bar(x of-type-X): pass
which, by means of overload resolution, will be found from foo1 automatically.
In Python, this is not so easy since you don't have overloading.
Regards, Martin
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev
===== -- S. Lott, CCP :-{) S_LOTT@YAHOO.COM http://www.mindspring.com/~slott1 Buccaneer #468: KaDiMa Macintosh user: drinking upstream from the herd. __________________________________________________ Do You Yahoo!? Yahoo! - Official partner of 2002 FIFA World Cup http://fifaworldcup.yahoo.com
In python, you don't need overloading, you have a variety of optional parameter mechanisms
...which forces users to write centralized dispatching mechanism that could be much more elegantly-handled by the language. The language already does something just for operators, but the rules are complicated and don't scale well.
I think the "member functions" issues from C++ don't apply to Python becuase C++ is strongly typed, meaning that many similar functions have to be written with slightly different type signatures.
That's very seldom the case in my C++ code. Why would you do that in lieu of writing function templates? I think Martin hit the nail on the head: you can achieve some decoupling of algorithms from data structures using free functions, but you need some way to look up the appropriate free function for a given data structure. FOr that, you need some kind of overload resolution.
The lack of strong typing makes it practical to write generic operations.
Templates and overloading in C++ make it practical to write statically-type-checked generic operations.
In python, you don't need overloading, you have a variety of optional parameter mechanisms
...which forces users to write centralized dispatching mechanism that could be much more elegantly-handled by the language. The language already does something just for operators, but the rules are complicated and don't scale well.
I don't think the situation can be improved without adding type declarations. --Guido van Rossum (home page: http://www.python.org/~guido/)
From: "Guido van Rossum" <guido@python.org>
In python, you don't need overloading, you have a variety of optional parameter mechanisms
...which forces users to write centralized dispatching mechanism that could be much more elegantly-handled by the language. The language already does something just for operators, but the rules are complicated and don't scale well.
I don't think the situation can be improved without adding type declarations.
You could do a little without type declarations, to handle functions with diffrent numbers of arguments or keywords, though I don't think that would be very satisfying. A good solution would not neccessarily need full type declaration capability; just some way to annotate function signatures with types. What I mean is that, for example, the ability to declare the type of a local variable would not be of any use in overload resolution. In fact, pure (sub)type comparison is probably not the best mechanism for Python's overload resolution. For example, it should be possible to write a function which will match any sequence object. I'd like to see something like the following sketch: 1. Each function can have an optional associated rating function which, given (args,kw) returns a float describing the quality of the match to its arguments 2. When calling an overloaded function, the best match from the pool of overloads is taken 3. The default rating function works as follows: a. Each formal argument has an optional associated match object b. The match object contributes a rating to the overall rating of the function c. If the match object is a type T, the rating system favors arguments x where T appears earlier in the mro of x.__class__. The availability of explicit conversions such as int() and float() is considered, but always produces a worse match than a subtype match. d. If the match object is a callable non-type, it's expected to produce an argument match rating Obviously, there are some details missing. I have to think about this stuff anyway for Boost.Python (since its current trivial overload resolution is not really adequate); if there's any interest here it would be nice for me, since I'd stand a chance of doing something which is likely to be consistent with whatever happens in Python. -Dave
In python, you don't need overloading, you have a variety of optional parameter mechanisms
...which forces users to write centralized dispatching mechanism that could be much more elegantly-handled by the language. The language already does something just for operators, but the rules are complicated and don't scale well.
I don't think the situation can be improved without adding type declarations.
I thought this was the issue interfaces were supposed to handle? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "In the end, outside of spy agencies, people are far too trusting and willing to help." --Ira Winkler
I thought this was the issue interfaces were supposed to handle?
You'd still need a way to attach an interface declaration to a function argument. Smell likes type declarations to me. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum <guido@python.org>:
This is funny. In the C++ community there's a nearly unanimous consensus that way too much of the functionality of the standard strings is expressed as member functions.
Interesting. Python used to have the same attitude, hence the string module -- but the existence of multiple string types made methods more attractive.
I don't understand how using methods makes multiple string types easier to deal with. Suppose I want to define my own string-like type. How do I arrange things so that " ".join(x) does the right thing when x contains instances of my new type? How is this made easier by the fact that join is a method of " " rather than a free function? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
I don't understand how using methods makes multiple string types easier to deal with.
Suppose I want to define my own string-like type. How do I arrange things so that " ".join(x) does the right thing when x contains instances of my new type? How is this made easier by the fact that join is a method of " " rather than a free function?
No solution in current Python deals really well with operations of two or more strings that may be of mixed types (the dispatch of binary operators comes closest). The current code checks for both standard string types to overcome this, and that doesn't work for a third, new string type. The point of methods was that homogeneous use (and all operations on a single string like s.lower() are homogeneous) is supported well. Because Python has no dispatch on argument type, the alternative would have been to rewrite the string module to do argument type tests in *all* functions -- and it would not support a third string type at all, not even when used homogeneous. IOW, if I write a function that calls s.lower() for an argument s, I can write a custom string type (e.g. UserString) that will work with f. If I wrote the same function using string.lower(s), I have no hope (short of modifying the string module). --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
IOW, if I write a function that calls s.lower() for an argument s, I can write a custom string type (e.g. UserString) that will work with f. If I wrote the same function using string.lower(s), I have no hope (short of modifying the string module).
unless the functions in the string module delegate everything to the object (just like len(), getattr(), and others), and you make sure that your type behaves well when mixed with basic strings. something like, say: def lower(s): return s.lower() or def capwords(s, sep=None): return join(map(capitalize, s.split(sep)), sep or ' ') Cheers /F
[Guido van Rossum]
[...] I don't think that the data objects have any other function except for membership testing [...]
This is useful being able to quickly enumerate letters, or digits. Just yesterday, I wrote: contexte = {} for nom, valeur in map(None, string.uppercase[:len(valeur)], valeur): assert valeur in '01', valeur contexte[nom] = valeur != '0' -- François Pinard http://www.iro.umontreal.ca/~pinard
participants (22)
-
Aahz -
Alex Martelli -
barry＠zope.com -
David Abrahams -
Fredrik Lundh -
Greg Ewing -
Guido van Rossum -
holger krekel -
M.-A. Lemburg -
martin＠v.loewis.de -
Michael Gilfix -
Neal Norwitz -
Neil Schemenauer -
Oren Tirosh -
Oren Tirosh -
Paul Prescod -
pf＠artcom-gmbh.de -
pinard＠iro.umontreal.ca -
Raymond Hettinger -
Skip Montanaro -
Steve Holden -
Steven Lott