Mailman 3 August 2002 - Python-Dev

convertibility and "Pythonicity"
by David Abrahams Aug. 24, 2002

Aug. 24, 2002

[Is "Pythonicity" the right word?] I'm interested in getting some qualitative feedback about something I'm doing in Boost.Python. The questions are, 1. How well does this behavior match up with what Python users have probably come to expect? 2. (related, I hope!) How close is it to the intended design of Python? When wrapping a C++ function that expects a float argument, I thought it would be bizarre if people couldn't pass a Python int. Well, Python ints have a lovely __float__ function which can be used to convert them to floats. Following that idea to its "logical" conclusion led me to where I am today: when matching a formal argument corresponding to one of the built-in Python types, first use the corresponding conversion slot. That could lead to some surprising behaviors: char index(const char* s, int n); // wrapped using Boost.Python >>> index('foobar', 2) # ok 'o' >>> index(3.14, 1.2) # Wierd (floats have __str__) '.' >>> index([1, 3, 5], 0.0) # Super wierd (everything has __str__) '[' So I went back and tried some "obvious" test in Python 2.2.1: >>> 'foobar'[3.0] Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: sequence index must be integer Well, I had expected this to work, so I'm beginning to re-think my "liberal conversion" policy. It seems like Python itself isn't using these slots to do "implicit conversion". But then: >>> 'foobar'[3L] 'b' [The int/long unification I've heard about hasn't happened yet, has it?] Furthermore: >>> range(3.3, 10.3) [3, 4, 5, 6, 7, 8, 9] But: >>> range('1', '5') Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: an integer is required Now I note that strings don't have __int__, so I guess the int type handles int('42') itself using special knowledge about strings. I suppose that's to keep strings from seeming to be numbers, since the nb_int slot fills in the number_methods. And: >>> class zero(object): ... def __int__(self): return 0 ... >>> range(zero(), 5) [0, 1, 2, 3, 4] So, is there any general practice, (even if it's not universal)? Do Python functions usually tend to coerce their arguments into the types they're expecting? I'm guessing the answer is no... ----------------------------------------------------------- David Abrahams * Boost Consulting dave(a)boost-consulting.com * http://www.boost-consulting.com

1 0

it seems we need types-sig back:)
by Samuele Pedroni Aug. 24, 2002

Aug. 24, 2002

.

1 0

utf8 issue
by Guido van Rossum Aug. 23, 2002

Aug. 23, 2002

This might beling on SF, except it's already been solved in Python 2.3, and I need guidance about what to do for Python 2.2.2. In 2.2.1, a lone surrogate encoded into utf8 gives an utf8 string that cannot be decode back. In 2.3, this is fixed. Should this be fixed in 2.2.2 as well? I'm asking because it caused problems with reading .pyc files: if there's a Unicode literal containing a lone surrogate, reading the .pyc file causes an exception: UnicodeError: UTF-8 decoding error: unexpected code byte It looks like revision 2.128 fixed this for 2.3, but that patch doesn't cleanly apply to the 2.2 maintenance branch. Can someone help? --Guido van Rossum (home page: http://www.python.org/~guido/)

1 0

type categories
by Samuele Pedroni Aug. 23, 2002

Aug. 23, 2002

some thoughts of mine [GvR] > Agreeing on an ontology seems the hardest part to me. Why does it seem such a daunting task? i) much of python code depends concretely on interfaces with the granularity from one to a bunch of methods, especially code expecting base-type-like implementations. ii) people appreciate to be able to implement just the minimal subset of methods that makes things work [Obviously here I'm not talking about large frameworks like Zope] We are not in vacuum. There is Python code out there, and progammers with ideas about what is like programming in Python. [Maybe I just restate the obvious and repeat myself but it seems that for some people not only type checking but in general explicitness about types is a tabu for Python code. OTOH there _exists_ Python code that as input depends/requires subclasses of some _specific_ abstract classes. And even Smalltalk has - when "reasonable" and "necessary" - a kind of interface notion, in the form of isFoo method defined on Object and overridden to return "yes sir" down the hierarchy] It seems to me that there is no route to the advantages of type categories without some explicitness. Now to the point. [Here my target is more dispatching and "distinguishing" by type categories than type checking, PEP 246 issue is far from orthogonal but here I will ignore it because I don't want my head to explode <wink>] Maybe it is obvious but anyway type categories' "problem" should be framed wrt: - what kind of coding style we want to enable (or maybe respectively deprecate)? - what problems are we solving? - what kind of code will not work anymore? - what "migration path" for such code, or to the new style? Let's consider an exemplar fragment: if hasattr(f,'write'): ... # needs just f.write else: # f is not file-like ... possible future styles: 1) if doesimplement(f,FileLike): ... else: ... problems: the "exponential" ontology problem, or the problem, if we limit ourselves only to large granularity interfaces and we interpret them strictly[*], that the programmer must implement more methods than strictly necessary. [*] the point is whether an interface should be interpreted as "all the signatures implemented" . Here is not relevant this is checked or enforced at class definition time. 2) if doesimplement(f,Category('write')): ... else: ... if we allow for such on-the-fly constructed interfaces (I hope one can extrapolate what I mean with that) we maybe solve the "exponential" ontology problem, but this code is not really an improvement over the code using hasattr; what we are interested in is not whether f has some kind of write method but whether f has a file-like write method. Through interfaces one wants to check and convey commitment more than at the signature level. Can we do better? It would be nice to find some kind of middle-ground between hasattr(.,'write') and large granularity strict interfaces (LGSI). My humble ideas: these are just two points picked in that entire range (hasattr - LSGI), other variations are maybe useful, necessary, or reasonable. a) As a "workaround" it should be possible to declare that a class implements an interface partially, that means some subset of it. Then it is an open issue whether there should be ways to check both for strict and non-strict implementation, or some general control to tweak all checkings and/or enable warnings. [Do Zope interfaces already allow for this?] b) OK with a) but if I want potentially to be strict and still deal with: "b) much of python code concretely depends on interfaces with the granularity from one to a bunch of methods" we should let people be precise about what subset of an interface they are implementing, like - I'm implemeting a subset of FileLike so consider the corresponding matching signatures - or, with finer control, I'm implementing FileLike 'write' and - no - my 'tell' has nothing to do with file-like ... and more importantly it should be possible to check for such subsets: if doesimplement(f,PartCategory(FileLike,['write'])): # Yup, I know, this is ugly and begs for sugar ... else: ... [And mildly interestingly such code can degrade to just check for hasattr(.,'write') for "migration" and possibly emit warnings] regards, Samuele Pedroni. PS: I know, here I have not dealt with implementation or performance problems and the killing details.

1 0

PEP 269 Implementation, rev.0
by Jonathan Riehl Aug. 23, 2002

Aug. 23, 2002

As per earlier discussions, I am going to take a whopping huge intermission that will run August right out, and end my yearly yernings to expose pgen to the Python public. Therefore, I've submitted a provisional patch for parser people to play with until next August (*smirk*). Get it while it's hot (ID 599331), and still in sync with CVS (not that there are any radical changes): http://sourceforge.net/tracker/index.php?func=detail&aid=599331&group_id=54… Comments are requested. Thanks! -Jon

1 0

More pydoc questions
by David Abrahams Aug. 23, 2002

Aug. 23, 2002

I recently added an invocation to help(my_extension_module) to the Boost.Python test suite, to prove that I can give reasonable help output. Worked great for me, since I was always running the test from within emacs. However, some other developer complained that the test required user intervention to run, since it would prompt at each screenful. So, I changed it to: print pydoc.TextDoc().docmodule(my_extension_module) Now I get (well, I'm not sure how this will show up in your mailer, but for me it's full of control characters): NNAAMMEE docstring_ext FFIILLEE c:\build\libs\python\test\bin\docstring_ext.pyd\vc7.1\debug\runtime-link-dy namic\docstring_ext.pyd DDEESSCCRRIIPPTTIIOONN A simple test module for documentation strings Exercised by docstring.py CCLLAASSSSEESS Boost.Python.instance(__builtin__.object) X class XX(Boost.Python.instance) | A simple class wrapper around a C++ int ... So my question is, is there a way to dump the text help for a module without prompting and without any extra control characters? TIA, Dave P.S. Another question: the docmodule() function takes two optional arguments whose role is undocumented. What are they for? ----------------------------------------------------------- David Abrahams * Boost Consulting dave(a)boost-consulting.com * http://www.boost-consulting.com

6 8

RE: [Python-checkins] python/dist/src/Lib pyclbr.py,1.26,1.27
by Tim Peters Aug. 23, 2002

Aug. 23, 2002

> Update of /cvsroot/python/python/dist/src/Lib > In directory usw-pr-cvs1:/tmp/cvs-serv15469 > > Modified Files: > pyclbr.py > Log Message: > Rewritten using the tokenize module, which gives us a real tokenizer > rather than a number of approximating regular expressions. > Alas, it is 3-4 times slower. Let that be a challenge for the > tokenize module. Was this just for purity, or did it fix a bug? The regexps there were close to being heroically careful, and even so it was somtimes uncomfortably slow using the class browser in IDLE (based on pyclbr), and even on a fast machine. A factor of 3 or 4 might make that unbearable. If it was for purity, note that tokenize is also based on mounds of regexp tricks <wink>.

2 1

q about default args
by Stepan Koltsov Aug. 23, 2002

Aug. 23, 2002

Hi, Guido, other Python developers and other subscribers. First of all, if this question was discussed here or somewhere else 8086 times, please direct me to discussion archives. I couldn't guess the keywords to search for in the python-dev archives as I haven't found the search page where to enter these keywords :-) The question is: To be or^H^H^H^H^H^H^H^H^H Why not evaluate default parameters of a function at THE function call, not at function def (as is done currenly)? For example, C++ (a nice language, isn't it? ;-) ) evaluates default parameters at function call. Example below illustrates the point in mind: In Python: ---------- def func(l = []): do_something(l) ========== to make it clearer (though this is not exactly the same): --- def func(l = list()): do_something(l) === or, in C++: --- void func(list l = list()) { do_something(l) } === Implementation details: Simple... Add a flag to the code object, that means "evaluate default args". Compile default args to small code objects and store them where values for default args are stored in current Python (i.e. co_consts). When a function is called, evaluate the default args (if the above flag is set) in the context of that function. So, for inst., this code is now possible: --- class Tree: # this iter walks through a Tree level-wise (i.e. left to right the down). def levelIter(self, nodes=[self]): # ^^^^^^ look here # the following is not "mission critical" :-) if len(nodes) == 0: return for node in nodes: yield node nodes = reduce(operator.add, [n.children() for n in nodes]) for node in self.levelIter(nodes): yield node === About compatibility: compiled python files stay backward compatible as long as they do not define the mentioned flag. An alternative way to go (a little example... LOOK ON, PERSONALY, I LIKE IT ALLOT): --- def f(x=12+[]): stmts === compiled into something like: 0: LOAD_CONST 1 (12) 1: BUILD_LIST 0 2: BINARY_ADD 3: STORE_FAST 0 (x) 4: # here code of stmts begin in the case if 'x' was specfied, the code is executed instruction 4 onword This should work perfectly, ideologically correct and I think even faster then current interpreter implementation. Motivation (he-he, the most difficult part of this letter): 1. Try to import this module: ---xx.py--- import math def func(a = map(lambda x: math.sqrt(x)): pass # there is no call to func === This code does nothing but define a single function, but look at the execution time... 2. Currently, default arguments are like static function variables, defined in the function parameter list! That is wrong. 4. Again: I dislike code like --- def f(l=None): if l is None: l = [] ... === 5. I asked my friend (also big Python fan): why the current behaviour is correct? his answer was: "the curren behaviour is correct, becausethat is the way it was done in the first place :-) ..." I don't see any advantages of the current style, and lack of advantages is advantage of new style :-) I hope, that the current state of things is a result of laziness (or is it "business"), not sabotage :-) . and not an ideological decision. It isn't late to fix Python yet :-) , as when Cpt. J. L. Picard once again saves the galaxy (this time, not from the evil Borg), it will be difficult to change self-modificating Python compilers, reconstruct hardware Python bytecode interpreters and verify tetrabytes of source code, written in Python (NOTE: I speek of the not so distant future :-) ) -- mailto: Stepan Koltsov <yozh(a)mx1.ru>

6 6

Re: [Python-Dev] q about default args
by Michael Hudson Aug. 22, 2002

Aug. 22, 2002

Stepan Koltsov <yozh(a)mx1.ru> writes: > Hi, Guido, other Python developers and other subscribers. > > > First of all, if this question was discussed here or somewhere > else 8086 times, please direct me to discussion archives. I doubt it's ever been discussed on python-dev. Most people here know a non-starter when they see one. Hmm. Well, they know this one's a non-starter :) > I couldn't guess the keywords to search for in the python-dev archives > as I haven't found the search page where to enter these keywords :-) Just use google. If python-dev is the most relavent place for the discussion, the archives will be near the top of the results. > The question is: To be or^H^H^H^H^H^H^H^H^H Why not evaluate default > parameters of a function at THE function call, not at function def > (as is done currenly)? For example, C++ (a nice language, isn't it? ;-) No. > ) evaluates default parameters at function call. > [...] > Implementation details: > > Simple... > Add a flag to the code object, that means "evaluate default args". > Compile default args to small code objects and store them where values > for default args are stored in current Python (i.e. co_consts). That's not where they're stored. > When a function is called, evaluate the default args (if the above > flag is set) in the context of that function. This could break code, you realise: a = 1 def f(a, b=a): print a, b [...] I could go on, but I'm running out of steam... > An alternative way to go (a little example... LOOK ON, PERSONALY, I > LIKE IT ALLOT): Fortunately or unfortunately, that makes little difference to the direction of python development. > --- > > def f(x=12+[]): > stmts > > === > > compiled into something like: > > 0: LOAD_CONST 1 (12) > 1: BUILD_LIST 0 > 2: BINARY_ADD > 3: STORE_FAST 0 (x) > 4: # here code of stmts begin > > in the case if 'x' was specfied, the code is executed instruction 4 > onword This should work perfectly, ideologically correct and I think > even faster then current interpreter implementation. You'd have fun with: def f(a=1,b=2): print a, b f(b=1) here, no? > > Motivation (he-he, the most difficult part of this letter): > > 1. Try to import this module: > > ---xx.py--- > > import math > def func(a = map(lambda x: math.sqrt(x)): > pass > # there is no call to func > > === > > This code does nothing but define a single function, > but look at the execution time... So don't do something that thick, then! > 2. Currently, default arguments are like static function variables, > defined in the function parameter list! That is wrong. Says you. > 4. Again: I dislike code like > > --- > > def f(l=None): > if l is None: > l = [] > ... Who elected you style guru of the universe? > 5. I asked my friend (also big Python fan): why the current > behaviour is correct? his answer was: "the curren behaviour is > correct, becausethat is the way it was done in the first place :-) > ..." I don't see any advantages of the current style, and lack of > advantages is advantage of new style :-) For better of for worse, people *do* write code that depends on default function arguments being evaluated once, usually as a lazy way of precomputing things, or as a cache. > I hope, that the current state of things is a result of laziness (or is > it "business"), not sabotage :-) . and not an ideological decision. It > isn't late to fix Python yet :-) Two points: 1) I'm unconvinced this is a "fix" 2) I think it probably is too late. Cheers, M. -- You can lead an idiot to knowledge but you cannot make him think. You can, however, rectally insert the information, printed on stone tablets, using a sharpened poker. -- Nicolai -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html

1 0

Re: [Python-Dev] Re: PEP 218 (sets); moving set.py to Lib
by Michael Hudson Aug. 22, 2002

Aug. 22, 2002

Greg Ewing <greg(a)cosc.canterbury.ac.nz> writes: > "Eric S. Raymond" <esr(a)thyrsus.com> > Subject: Re: [Python-Dev] Re: PEP 218 (sets): > > > Is it + for disjunction and juxtaposition for conjunction, or the other > > way around? > > + is 'or' and juxtaposition (or sometimes a dot) is 'and' > (I prefer those words because they're shorter than > 'disjunction' and 'conjunction', and I can remember which > is which:-). I've always thought "meet" and "join" to be quite cute terms for lattice operations. Guaranteed to confuse the average user of sets.py -- let's go for it! Cheers, M. -- The use of COBOL cripples the mind; its teaching should, therefore, be regarded as a criminal offence. -- Edsger W. Dijkstra, SIGPLAN Notices, Volume 17, Number 5

1 0