I noticed that UserString objects have methods that do not accept other UserString objects as arguments:
from UserString import UserString UserString('slartibartfast').count(UserString('a'))
Traceback (most recent call last): File "<pyshell#2>", line 1, in -toplevel- UserString('slartibartfast').count(UserString('a')) File "C:\PY24\lib\UserString.py", line 66, in count return self.data.count(sub, start, end) TypeError: expected a character buffer object
UserString('abc') in UserString('abcde')
Traceback (most recent call last): File "<pyshell#1>", line 1, in -toplevel- UserString('abc') in UserString('abcde') File "C:\PY24\lib\UserString.py", line 35, in __contains__ return char in self.data TypeError: 'in <string>' requires string as left operand This sort of thing is easy to test for and easy to fix. The question is whether we care about updating this module anymore or is it a relic. Also, is the use case one that we care about. AFAICT, this has never come up before. Raymond
On 2005 Feb 20, at 05:20, Raymond Hettinger wrote: ...
This sort of thing is easy to test for and easy to fix. The question is whether we care about updating this module anymore or is it a relic. Also, is the use case one that we care about. AFAICT, this has never come up before.
I did have some issues w/UserString at a client's, but that was connected to some code doing type-checking (and was fixed by injecting basestring as a base of the client's subclass of UserString and ensuring the type-checking always used isinstance and basestring). My two cents: a *mixin* to make it easy to emulate full-fledged strings would be almost as precious as your DictMixin (ones to emulate lists, sets, files [w/buffering], ..., might be even more useful). The point is all of these rich interfaces have a lot of redundancy and a mixin can provide all methods generically based on a few fundamental methods, which can be quite useful, just like DictMixin. But a complete emulation of strings (etc) is mostly of "didactical" use, a sort of checklist to help ensure one implements all methods, not really useful for new code "in production"; at least, I haven't found such uses recently. The above-mentioned client's class was an attempt to join RE functionality to strings and was a rather messy hack anyway, for example (perhaps prompted by client's previous familiarity with Perl, I'm not sure); at any rate, the client should probably have subclassed str or unicode if he really wanted that hack. I can't think of a GOOD use for UserString (etc) since subclassing str (etc) was allowed in 2.2 or at least since a few loose ends about newstyle classes were neatly tied up in 2.3. If we do decide "it is a relic, no more updates" perhaps some indication of deprecation would be warranted. ((In any case, I do think the mixins would be useful)). Alex
[Alex]
I did have some issues w/UserString at a client's, but that was connected to some code doing type-checking (and was fixed by injecting basestring as a base of the client's subclass of UserString and ensuring the type-checking always used isinstance and basestring).
Oh, bah. That's not what basestring was for. I can't blame you or your client, but my *intention* was that basestring would *only* be the base of the two *real* built-in string types (str and unicode). The reason for its existence was that some low-level built-in (or extension) operations only accept those two *real* string types and consequently some user code might want to validate ("look before you leap") its own arguments if those eventually ended up being passed to aforementioned low-level built-in code. My intention was always that UserString and other string-like objects would explicitly *not* inherit from basestring. Of course, my intention was lost, your client used basestring to mean "any string-ish object", got away with it because they weren't using any of those low-level built-ins, and you had to comply rather than explain it to them. Sounds like a good reason to add interfaces to the language. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On 2005 Feb 20, at 17:06, Guido van Rossum wrote:
[Alex]
I did have some issues w/UserString at a client's, but that was connected to some code doing type-checking (and was fixed by injecting basestring as a base of the client's subclass of UserString and ensuring the type-checking always used isinstance and basestring).
Oh, bah. That's not what basestring was for. I can't blame you or your client, but my *intention* was that basestring would *only* be the base of the two *real* built-in string types (str and unicode). The reason for its existence was that some low-level built-in (or extension) operations only accept those two *real* string types and consequently some user code might want to validate ("look before you leap") its own arguments if those eventually ended up being passed to aforementioned low-level built-in code. My intention was always that UserString and other string-like objects would explicitly *not* inherit from basestring. Of course, my intention was lost, your client used basestring to mean "any string-ish object", got away with it because they weren't using any of those low-level built-ins, and you had to comply rather than explain it to them.
I would gladly have explained, if I had understood your design intent correctly at the time (whether the explanation would have done much good is another issue); but I'm afraid I didn't. Now I do (thanks for explaining!) though I'm not sure what can be done in retrospect to communicate it more widely. The need to check "is this thingy here string-like" is sort of frequent, because strings are sequences which, when iterated on, yield sequences (strings of length 1) which, when iterated on, yield sequences ad infinitum. Strings are sequences but more often than not one wants to treat them as "scalars" instead. isinstance and basestring allow that frequently needed check so nicely, that, if they're not intended for it, they're an "attractive nuisance" legally;-). The need to make stringlike thingies emerges both for bad reasons (e.g., I never liked that client's "string cum re" perloidism) and good ones (e.g., easing the interfacing with external frameworks that have their own stringythings, such as Qt's QtString); and checking if something is stringlike is also frequent, as per previous para. Darn...
Sounds like a good reason to add interfaces to the language. :-)
If an interface must be usable to say "is this string-like?" it will have to be untyped, I guess, and the .translate method will be a small problem (one-argument for unicode, two-args for str, and very different argument semantics) -- don't recall offhand if there are other such nonpolymorphic methods there. Alex
Alex Martelli wrote:
On 2005 Feb 20, at 17:06, Guido van Rossum wrote:
Oh, bah. That's not what basestring was for. I can't blame you or your client, but my *intention* was that basestring would *only* be the base of the two *real* built-in string types (str and unicode).
I think all this just reinforces the notion that LBYL is a bad idea!
The need to check "is this thingy here string-like" is sort of frequent, because strings are sequences which, when iterated on, yield sequences (strings of length 1) which, when iterated on, yield sequences ad infinitum.
Yes, this characteristic of strings is unfortunate because it tends to make some degree of LBYLing unavoidable. I don't think the right solution is to try to come up with safe ways of doing LBYL on strings, though, at least not in the long term. Maybe in Python 3000 this could be fixed by making strings *not* be sequences. They would be sliceable, but *not* indexable or iterable. If you wanted to iterate over their chars, you would have to say 'for c in s.chars()' or something. Then you would be able to test whether something is sequence-like by the presence of __getitem__ or __iter__ methods, without getting tripped up by strings. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+
Oh, bah. That's not what basestring was for. I can't blame you or your client, but my *intention* was that basestring would *only* be the base of the two *real* built-in string types (str and unicode).
I think all this just reinforces the notion that LBYL is a bad idea!
In this case, perhaps; but in general? (And I think there's a legitimate desire to sometimes special-case string-like things, e.g. consider a function that takes either a stream or a filename argument.) Anyway, can you explain why LBYL is bad?
The need to check "is this thingy here string-like" is sort of frequent, because strings are sequences which, when iterated on, yield sequences (strings of length 1) which, when iterated on, yield sequences ad infinitum.
Yes, this characteristic of strings is unfortunate because it tends to make some degree of LBYLing unavoidable. I don't think the right solution is to try to come up with safe ways of doing LBYL on strings, though, at least not in the long term.
Maybe in Python 3000 this could be fixed by making strings *not* be sequences. They would be sliceable, but *not* indexable or iterable. If you wanted to iterate over their chars, you would have to say 'for c in s.chars()' or something.
Then you would be able to test whether something is sequence-like by the presence of __getitem__ or __iter__ methods, without getting tripped up by strings.
There would be other ways to get out of this dilemma; we could introduce a char type, for example. Also, strings might be recognizable by other means, e.g. the presence of a lower() method or some other characteristic method that doesn't apply to sequence in general. (To Alex: leaving transform() out of the string interface seems to me the simplest solution.) -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On 2005 Feb 21, at 04:42, Guido van Rossum wrote:
Oh, bah. That's not what basestring was for. I can't blame you or your client, but my *intention* was that basestring would *only* be the base of the two *real* built-in string types (str and unicode).
I think all this just reinforces the notion that LBYL is a bad idea!
In this case, perhaps; but in general? (And I think there's a legitimate desire to sometimes special-case string-like things, e.g. consider a function that takes either a stream or a filename argument.)
Anyway, can you explain why LBYL is bad?
In the general case, it's bad because of a combination of issues. It may violate "once, and only once!" -- the operations one needs to check may basicaly duplicate the operations one then wants to perform. Apart from wasted effort, it may happen that the situation changes between the look and the leap (on an external file, or due perhaps to threading or other reentrancy). It's often hard in the look to cover exactly the set of prereq's you need for the leap -- e.g. I've often seen code such as if i < len(foo): foo[i] = 24 which breaks for i<-len(foo); the first time this happens the guard's changed to 0<=i<len(foo) which then stops the code from working w/negative index; finally it stabilizes to the correct check, -len(foo)<=i<len(foo), but even then it's just duplicating the same check that Python performs again when you then use foo[i]... just cluttering code. The intermediate Pythonista's who's learned to code "try: foo[i]=24 // except IndexError: pass" is much better off than the one who's still striving to LBYL as he had (e.g.) when using C. Etc -- this is all very general and generic. I had convinced myself that strings were a special case worth singling out, via isinstance and basestring, just as (say) dictionaries are singled out quite differently by metods such as get... I may well have been too superficial in this conclusion.
Then you would be able to test whether something is sequence-like by the presence of __getitem__ or __iter__ methods, without getting tripped up by strings.
There would be other ways to get out of this dilemma; we could introduce a char type, for example. Also, strings might be recognizable by other means, e.g. the presence of a lower() method or some other characteristic method that doesn't apply to sequence in general.
Sure, there would many possibilities.
(To Alex: leaving transform() out of the string interface seems to me the simplest solution.)
I guess you mean translate. Yes, that would probably be simplest. Alex
Anyway, can you explain why LBYL is bad?
In the general case, it's bad because of a combination of issues. It may violate "once, and only once!" -- the operations one needs to check may basicaly duplicate the operations one then wants to perform. Apart from wasted effort, it may happen that the situation changes between the look and the leap (on an external file, or due perhaps to threading or other reentrancy). It's often hard in the look to cover exactly the set of prereq's you need for the leap -- e.g. I've often seen code such as if i < len(foo): foo[i] = 24 which breaks for i<-len(foo); the first time this happens the guard's changed to 0<=i<len(foo) which then stops the code from working w/negative index; finally it stabilizes to the correct check, -len(foo)<=i<len(foo), but even then it's just duplicating the same check that Python performs again when you then use foo[i]... just cluttering code. The intermediate Pythonista's who's learned to code "try: foo[i]=24 // except IndexError: pass" is much better off than the one who's still striving to LBYL as he had (e.g.) when using C.
Etc -- this is all very general and generic.
Right. There are plenty of examples where LBYL is better, e.g. because there are too many different exceptions to catch, or they occur in too many places. One of my favorites is creating a directory if it doesn't already exist; I always use this LBYL-ish pattern: if not os.path.exists(dn): try: os.makedirs(dn) except os.error, err: ...log the error... because the specific exception for "it already exists" is quite subtle to pull out of the os.error structure. Taken to th extreme, the "LBYL is bad" meme would be an argument against my optional type checking proposal, which I doubt is what you want. So, I'd like to take a much more balanced view on LBYL.
I had convinced myself that strings were a special case worth singling out, via isinstance and basestring, just as (say) dictionaries are singled out quite differently by metods such as get... I may well have been too superficial in this conclusion.
I think there are lots of situations where the desire to special-case strings is legitimate.
Then you would be able to test whether something is sequence-like by the presence of __getitem__ or __iter__ methods, without getting tripped up by strings.
There would be other ways to get out of this dilemma; we could introduce a char type, for example. Also, strings might be recognizable by other means, e.g. the presence of a lower() method or some other characteristic method that doesn't apply to sequence in general.
Sure, there would many possibilities.
(To Alex: leaving transform() out of the string interface seems to me the simplest solution.)
I guess you mean translate. Yes, that would probably be simplest.
Right. BTW, there's *still* no sign from a PEP 246 rewrite. Maybe someone could offer Clark a hand? (Last time I inquired he was recovering from a week of illness.) -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Mon, 2005-02-21 at 11:15, Guido van Rossum wrote:
Right. There are plenty of examples where LBYL is better, e.g. because there are too many different exceptions to catch, or they occur in too many places. One of my favorites is creating a directory if it doesn't already exist; I always use this LBYL-ish pattern:
if not os.path.exists(dn): try: os.makedirs(dn) except os.error, err: ...log the error...
because the specific exception for "it already exists" is quite subtle to pull out of the os.error structure.
Really? I do this kind of thing all the time: import os import errno try: os.makedirs(dn) except OSError, e: if e.errno <> errno.EEXIST: raise -Barry
Really? I do this kind of thing all the time:
import os import errno try: os.makedirs(dn) except OSError, e: if e.errno <> errno.EEXIST: raise
You have a lot more faith in the errno module than I do. Are you sure the same error codes work on all platforms where Python works? It's also not exactly readable (except for old Unix hacks). -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Tue, 22 Feb 2005 08:16:52 -0800, Guido van Rossum <gvanrossum@gmail.com> wrote:
Really? I do this kind of thing all the time:
import os import errno try: os.makedirs(dn) except OSError, e: if e.errno <> errno.EEXIST: raise
You have a lot more faith in the errno module than I do. Are you sure the same error codes work on all platforms where Python works? It's also not exactly readable (except for old Unix hacks).
Agreed. In general, I often wish in production code (especially in not-100% Python systems) that Python did a better job of at the very least documenting what kinds of exceptions were raised by what function calls. Otherwise you end up with what are effectively blanket try/except statements way too often for my taste. --da
if e.errno <> errno.EEXIST: raise
You have a lot more faith in the errno module than I do. Are you sure the same error codes work on all platforms where Python works? It's also not exactly readable (except for old Unix hacks).
On the other hand, LBYL in this context can result in race conditions and security vulnerabilities. "os.makedirs" is already a composite of many system calls, so all bets are off anyway, but for simpler operations that result in an atomic system call, this is important. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/
On Tue, 2005-02-22 at 19:14, Andrew McNamara wrote:
if e.errno <> errno.EEXIST: raise
You have a lot more faith in the errno module than I do. Are you sure the same error codes work on all platforms where Python works? It's also not exactly readable (except for old Unix hacks).
On the other hand, LBYL in this context can result in race conditions and security vulnerabilities. "os.makedirs" is already a composite of many system calls, so all bets are off anyway, but for simpler operations that result in an atomic system call, this is important.
Agreed. -Barry
On Tue, 2005-02-22 at 11:16, Guido van Rossum wrote:
Really? I do this kind of thing all the time:
import os import errno try: os.makedirs(dn) except OSError, e: if e.errno <> errno.EEXIST: raise
You have a lot more faith in the errno module than I do. Are you sure the same error codes work on all platforms where Python works?
No, but I'm pretty confident the symbolic names for the errors are consistent for any platform I've cared about <wink>.
It's also not exactly readable (except for old Unix hacks).
Guilty as charged. ;) -Barry
You have a lot more faith in the errno module than I do. Are you sure the same error codes work on all platforms where Python works?
No, but I'm pretty confident the symbolic names for the errors are consistent for any platform I've cared about <wink>.
It's also not exactly readable (except for old Unix hacks).
Guilty as charged. ;)
The consistency of the semantics of core system calls is sort of trademark of unix. Any system that claims to be Unix, but plays loose and fast with semantics soon gets a very poor reputation (xenix, cough). All well-coded unix apps are dependent on system calls returning consistent errno's. Which is one thing that makes life so difficult for "posix" environments layered on other operating systems. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/
At 04:32 PM 2/21/05 +1300, Greg Ewing wrote:
Alex Martelli wrote:
The need to check "is this thingy here string-like" is sort of frequent, because strings are sequences which, when iterated on, yield sequences (strings of length 1) which, when iterated on, yield sequences ad infinitum.
Yes, this characteristic of strings is unfortunate because it tends to make some degree of LBYLing unavoidable.
FWIW, the trick I usually use to deal with this aspect of strings in recursive algorithms is to check whether the current item of an iteration is the same object I'm iterating over; if so, I know I've descended into a string. It doesn't catch it on the first recursion level of course (unless it was a 1-character string to start with), but it's a quick-and-dirty way to EAFP such algorithms.
participants (8)
-
Alex Martelli -
Andrew McNamara -
Barry Warsaw -
David Ascher -
Greg Ewing -
Guido van Rossum -
Phillip J. Eby -
Raymond Hettinger