Mailman 3 February 2003 - Python-Dev

Re: [Python-checkins] python/nondist/peps pep-0311.txt,NONE,1.1
by Just van Rossum 14 Feb '03

14 Feb '03

mhammond(a)users.sourceforge.net wrote: > Open Issues > This is where I note comments from people that are yet to be resolved. > - JustvR prefers a PyGIL prefix over PyAutoThreadState. > - JackJ notes that the "Auto" prefix will look a little silly > in a few years, assuming this becomes the standard way of > managing the lock. He doesn't really like Just's "GIL", and > suggested "PyIntLock" Oh, so now we're locking integers? ;-) The reason I don't like "Auto" is that there's nothing automatic going on: you still have to _use_ the API actively... My opinion is biased because very recently I wrote a module called "autoGIL" for OSX, which actually _does_ handle the GIL automatically, in the context of event loops: it releases the GIL when the event loop goes to sleep and acquires it when it wakes up again. This works for us as the even loop is not supposed to use blocking calls to begin with. Handling the GIL the "proper" way in this project (PyObjC) would be impossible without PEP 311, but even with it, it will be an huge amount of work. I like PyGIL for the prefix as "GIL" is a well understood acronym; you can't really discuss the issues without using it... But it's totally your call. Just

1 0

Updated AutoThreadState pre-PEP
by Mark Hammond 14 Feb '03

14 Feb '03

Hi all, I have updated my pre-PEP, and am about to start the process of making it a real PEP :) I have attached the latest and greatest version here. I believe it largely complete. There is a fully functional, platform independent implementation at http://www.python.org/sf/684256 All comments, reviews, tests and anything else welcome. Mark.

1 1

PEP-242 Numeric kinds -- disposition
by Paul Dubois 14 Feb '03

14 Feb '03

PEP-242 should be closed. The kinds module will not be added to the standard library. There was no opposition to the proposal but only mild interest in using it, not enough to justify adding the module to the standard library. Instead, it will be made available as a separate distribution item at the Numerical Python site. At the next release of Numerical Python, it will no longer be a part of the Numeric distribution.

1 0

RE: [Python-Dev] Updated AutoThreadState pre-PEP
by Mark Hammond 14 Feb '03

14 Feb '03

[Aahz, in private mail, provokes an interesting point] > "What happens if you do PyAutoThreadState_Ensure() and > PyEval_InitThreads() has not been called?" > > There should be some discussion of the appropriate time to call > PyEval_InitThreads(). Yeah, this is an interesting, indirect point. If the author must call PyEval_InitThreads(), then they must also have called Py_Initialize(). If in the real world, all users of this API have already done this at process startup, then our internal checks for initialization are pointless. Therefore, I propose that I change the PEP to state that before the AutoThreadState API can be used, Python must have been "correctly initialized", where "correctly initialized" is defined as: * Py_Initialize() has been called. * If the extension is multi-threaded, then PyEval_InitThreads() has been called from the "main thread". If these conditions are not met, Python will either (and hopefully usually) Py_FatalError(), or occasionally dead-lock. Any objections? Mark.

1 0

dict "setdefault". Feaure request or bugfix?
by Jesus Cea Avion 13 Feb '03

13 Feb '03

<https://sourceforge.net/tracker/index.php?func=detail&aid=684542&group_id=5…> Reading "python cookbook" I reach a very intelligent dictionary construction: to use dictionary "setdefault" to avoid a "get" or a "try...except". Nevertheless current "setdefault" behaviour is questionable, since it evaluates the default value when it is not necessary. In my mind, "setdefault" should evaluate the default value when it need to use it, not ALWAYS. Example: >>> a={} >>> def b() : ... print "hi!" ... return 1 ... >>> a.setdefault("hola","hello") hello >>> print a {'hola': 'hello'} >>> a.setdefault("hola",b()) hi! 'hello' In the example, the "setdefault" evaluation is not necessary since 'a["hola"]' already exists. But the default function is called, although its return value will be ignored. This behaviour is not intuitive and I think it should be changed or, at least, documented. The change would be very welcome when the "setdefault" default code is expensive, like my application. In any case, the change would be ALWAYS more efficient, even in the tipical "[]" case. Perhaps a "form __future__ import setdefaultNG" would be necessary to help to update legacy code? Opinions?. -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea(a)argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ _/_/_/_/_/ PGP Key Available at KeyServ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz

9 13

Re: [Python-Dev] when is binary mode required for pickle?
by Gareth McCaughan 13 Feb '03

13 Feb '03

Guido wrote: > For most people, yes. "bin" is deprecated, "proto" should not be named > (cPickle doesn't accept keyword arguments), and passing -1 is the best thing > to do if you don't care about reading a pickle under an older version of > Python than the one that wrote the pickle. For compatibility reasons, > though, the default proto value had to remain 0. Suggestion: pickle.LATEST_PROTOCOL = cPickle.LATEST_PROTOCOL = -1 ? (It could be spelled differently.) Or the string value "latest" could be an acceptable value of proto. Or something. Anything rather than magic numbers. :-) -- g

2 1

Change definition of Py_END_ALLOW_THREADS?
by Mark Hammond 13 Feb '03

13 Feb '03

Hi all, As part of my "auto thread state" work, I am discovering some very interesting things - including the fact that pythread.h *already* defines a Python Thread Local Storage API, and that thread_sgi.h contains a safe, portable working implementation :) However, to do this auto thread state work properly, I will need to change the definition of Py_END_ALLOW_THREADS. Specifically, Py_END_ALLOW_THREADS needs to be able to handle the fact that the lock may or may not be held when it is called (whereas now, the lock must *not* be held) Specifically, the changes would look like: * New function to determine if the current thread's threadstate is "current" * New function, PyEval_EnsureThread(), which is nearly identical to PyEval_RestoreThread(), except it only attempts the restore of the thread state if it is not already current. The above 2 are fairly easy to implement in a thread-safe manner - if the thread only checks its *own* thread state is current, the result of that check will remain valid on this thread, even if another thread becomes current during processing. This means that Py_END_ALLOW_THREADS then changes from: #define Py_END_ALLOW_THREADS PyEval_RestoreThread(_save); \ to: #define Py_END_ALLOW_THREADS PyEval_EnsureThread(_save); \ Now, I understand that a complete blessing of this scheme will need more details - but I am concerned that *any* change to Py_END_ALLOW_THREADS will blow this scheme out of the water. For example, the Win32 extensions have, in a few places, emulated Py_BEGIN_ALLOW_THREADS, and all of this code will need to change. From my POV though, the only reason I needed these hacks in the first place was due to an incomplete thread state API, so such changes are worthwhile to me. Other's mileage may vary. FWIW, I have a portable (ie, no windows specific code at all) "auto thread state" implementation fully working with the Win32all extensions. These extensions do some funky things with threads and thread-states, and it all works perfectly. However, it does rely on this Py_END_ALLOW_THREAD change. See the full patch at http://www.python.org/sf/684256 Thanks, Mark.

5 14

when is binary mode required for pickle?
by Skip Montanaro 13 Feb '03

13 Feb '03

(I'm really not asking how to use pickle.) Unused as I am to programming on Windows, it's not clear to me under what circumstances I might need to open a file in binary mode for Pickle's consumption. The reason I ask is that while working on the csv PEP and module, we had some input complaining about our requirement that CSV files be opened with the 'b' flag (where it matters). Andrew mentioned that Pickle requires files be opened in binary mode. However, the documentation doesn't explicitly state this. Must, for example, a file be opened with the 'b' flag if an ASCII pickle is to be written? The doc for the Unpickler class says the file-like object must support both read(n) and readline() methods. The requirement for readline() suggests the file be opened in text mode. (Maybe this is just a nit with the Pickle docs and I don't know it?) Skip

4 4

Overly creative config in socketmodule.c
by Tim Peters 13 Feb '03

13 Feb '03

Someone very recently checked in this change: static PyObject* socket_inet_aton(PyObject *self, PyObject *args) { #ifndef INADDR_NONE #define INADDR_NONE (-1) #endif /* Have to use inet_addr() instead */ char *ip_addr; #if 1 ... if (inet_aton(ip_addr, &buf)) return PyString_FromStringAndSize((char *)(&buf), ... sizeof(buf)); #else /* In case you don't have inet_aton() */ ... #endif } Now in case you were wondering, #if 1 #else #endif isn't a robust way to do platform configuration <wink/arghghgh>. Whoever did this, please back it out or do it right. inet_aton doesn't exist on Windows, so socketmodule.c no longer compiles there.

2 2

SF CVS down?
by Brett Cannon 13 Feb '03

13 Feb '03

Or is just me? -Brett

2 2

[meta] EIBTI: can we acronimize?
by Lalo Martins 13 Feb '03

13 Feb '03

I just realized I'm finally tired of typing "Explicit is better than implicit". Can we agree to call this maxim EIBTI? Or is this too implicit? ;-) []s, |alo +---- -- Those who trade freedom for security lose both and deserve neither. -- http://www.laranja.org/ mailto:lalo@laranja.org pgp key: http://www.laranja.org/pessoal/pgp Eu jogo RPG! (I play RPG) http://www.eujogorpg.com.br/ GNU: never give up freedom http://www.gnu.org/

3 2

RE: some data on the debate [Python-Dev] Update to PEP308: if-then-else expression
by Chermside, Michael 12 Feb '03

12 Feb '03

> The relevant data for me was that 1/4, 1/2 of the total posts were made by just > a small "vocal" group of people Please don't think I missed the importance of this fact. The details of the count were unimportant compared to what this says about how the perceived level of vigorous argument may actually be an artifact of a few outliers. I actually only posted thinking that there might be a bug in your counting algorithm... I still believe that in the end Guido will wind up finding a way to extract a reasonable decision from this mess, although I have a hard time seeing how he'll manage that! Knowing that a few individuals account for a lot of the "noise" may help him. -- Michael Chermside

2 1

Codecs data
by Gustavo Niemeyer 12 Feb '03

12 Feb '03

I belive we have a problem in our current codecs implementation: it doesn't work for multiple interpreter states, because the data is currently statically allocated in codecs.c. That way, only the first interpreter is going to have the codecs lookup working. The following interpreters will try to cross the interpreter state boundary and use the lookup function from the first interpreter, issuing all kinds of weird errors. One solution would be to store that data in the "sys" module (e.g. codec_search_path, codec_search_cache, codec_inited). Can you please advice on how to fix that? I'll be willing to provide a fix with the proposed solution. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ]

3 5

Re: some data on the debate [Python-Dev] Update to PEP308: if-then-else expression
by Chermside, Michael 12 Feb '03

12 Feb '03

[Samuele Pedroni creates list of # of posts per person] I'm not quite sure how you generated this list, but I notice that I'm not on it. Now I *HAVE* been trying very hard to make FEW posts (but hopefully insightful ones), but I still made it to the threshold of 3. I don't particularly care about the list, but if Guido wants a listing of who's pro/con, then we shouldn't exclude people who have tried to REDUCE the amount of noise by posting a SMALL number of times. -- Michael Chermside

2 1

str.join, string.join
by Christian Tismer 12 Feb '03

12 Feb '03

Hi Tim, pydev, while fighting this conditional super thread, I stumbled over something about "".join, as it was used as an argument for reading LtoR. Citing A. Koenig and Ahz: """ ','.join(lines) Aahz> Many people (including me) think it's *NOT* Pythonic. If I were trying to make code easy to understand for inexperienced programmers, I would write str.join(',', lines) instead of ','.join(lines). """ I have two observations: 1) Through introduction of strings as class-like objects, new calling patterns are generated. Unfortunately, this is the opposite as in the string module: str.join(',', lines) # vs. string.join(lines, ',') What is now pythonic. Was it (a posteriori) a mistake to add join as a method to strings, since it creates a calling pattern that is incompatible with the string function? Side remark: This error message is confusing: >>> str.join(" ") Traceback (most recent call last): File "<interactive input>", line 1, in ? TypeError: join() takes exactly one argument (0 given) Well, the method is probably not well suited to be called by the class, only (expecting to be a method, always). But now it exists. 2) Given the str "class", we could use it to make the string module absolutely redundant, since we can put all string constants into str now. So we would be able to use str.digits # instead of string.digits Trying to make sense of 1) and 2), I get into trouble because of the inconsistent join issue. What is the long-term way to make this pythonic again: Should "".join be dropped in favor of a class method which reads like the old string.join? Or should it be kept like today, exist as "".join(stuff) and str.join("", stuff), and string.join becomes deprecated? ciao - chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

4 3

Re: Gadfly in Python core
by Aaron Watters 12 Feb '03

12 Feb '03

Hi Guido, Stuart, python-dev, Stuart was nice enough to forward this information to me and I would be delighted. I don't know exactly how to "transfer" a copyright, but I'd be happy to do it. Thanks for the interest! -- Aaron Watters (ps: I would have sent a more appropriate email yesterday, Stuart, but I was snowed under with work) Stuart Bishop wrote: > Hi. > > I don't know if you saw this thread or not on Python-dev: > http://mail.python.org/pipermail/python-dev/2003-January/032295.html > > Looks like Gadfly can go into the core Python distribution. One of the > outstanding issues is that of the licence, which concerned Guido: > http://mail.python.org/pipermail/python-dev/2003-January/032305.html > > Would you be interested in allowing the PSF to distribute Gadfly under > its own licence to simplify things? >

1 0

Re: GadflyDA in core? Or as add-on-product?
by Stuart Bishop 11 Feb '03

11 Feb '03

On Wednesday, January 22, 2003, at 08:22 AM, Guido van Rossum wrote: >>>> My personal belief would be to include Gadfly in Python: >>>> - Provides a reason for the DB API docs to be merged into the >>>> Python library reference >>>> - Gives Python relational DB stuff out of the box ala Java, >>>> but with a working RDBMS as well ala nothing else I'm aware >>>> of. >>>> - Makes including GadflyDA in Zope 3 a trivial decision, since >>>> its size would be negligable and the DA code itself is >>>> already ZPL. >>> >>> Would you be willing to find out (from c.l.py) how much interest >>> there is in this? >> >> A fairly positive response from the DB SIG. The trick will be to fix >> the outstanding bugs or disable those features (losing the 'group >> by' and 'unique' SQL clauses), and to confirm and fix any departures >> from the DB-API 2.0 standard, as this would become a reference >> implementation of sorts. >> >> There is no permanent maintainer, as Richard Jones is in more of a >> caretaker role with the code. I'll volunteer to try and get the code >> into a Python release though. >> >> If fixes, documentation and tests can be organized by the end of >> January for alpha2, will this go out with Python 2.3 (assuming a >> signoff on quality by python-dev and the DB-SIG)? If not, Jim is >> back to deciding if he should include Gadfly with Zope3. > > Sorry for not responding before. I'm open for doing this, but you > should probably probe python-dev next before you start a big coding > project. How much C code is involved in Gadfly? If it's a lot, I'm a > lot more reluctant, because C code usually requires much more > maintenance (rare is the C source file that doesn't have some hidden > platform dependency). Gadfly comes with kjbuckets, which is written in C. The rest is Python. Gadfly uses the included kjbuckets for storage if it is available, but happily runs without it with a performance hit. So Jython gets a RDBMS implementation too. -- Stuart Bishop <zen(a)shangri-la.dropbear.id.au> http://shangri-la.dropbear.id.au/

6 17

pickle me, Elmo?
by Paul F Dubois 11 Feb '03

11 Feb '03

I read the new PEP about pickling with much interest and little comprehension. I'd like to comment then just by saying what I wish I could do and invite response as to whether the PEP will help do it. I would like to be able to: Save and restore a class definition Save and restore a function definition

6 9

RE: [Python-checkins] python/dist/src/Python bltinmodule.c,2.276,2.277
by Tim Peters 11 Feb '03

11 Feb '03

[Jack Jansen] > ... > If you build with Apple's MPW compilers it probably is, but (a) these > compilers are horribly outdated and (b) it would take major surgery to > compile Python with MPW anyway, nowadays. In that case, can we remove the MPW #ifdef's from the Python core? Things like object.c's #ifdef MPW /* MPW C modf expects pointer to extended as second argument */ { extended e; fractpart = modf(v, &e); intpart = e; } #else fractpart = modf(v, &intpart); #endif and mathmodule.c's flea on the tip of the tail of the dog #ifdef MPW_3_1 /* This hack is needed for MPW 3.1 but not for 3.2 ... */ FUNC2(pow, power, "pow(x,y)\n\nReturn x**y (x to the power of y).") #else and acceler.c's germ on the aforemetioned flea's hindquarters: #ifdef MPW_881_BUG /* In 881 mode MPW 3.1 has a code generation bug which seems to set the upper bits; fix this by explicitly masking them off */ int temp; #endif And so on. These don't really improve maintainability <wink>.

2 1

vox populii illiterati
by Paul F Dubois 11 Feb '03

11 Feb '03

I very much appreciate Guido putting forward a PEP on if expressions, but I must say the idea of letting c.l.p 'vote' on it is unwelcome. Voting only makes sense when an electorate is educated enough to properly understand the proposition and its consequences; the discussion so far makes clear that that isn't true. Programming languages need to be designed, not agglutinated. (Otherwise you get Perl and Fortran 2000 (:->) I think it is fair to say that being the head designer of a language can lead to frustration at times; I had to endure it on a much smaller scale and it was very hard to keep patiently explaining how the features had to fit together, and how they had to be both parseable and implementable and learnable, and that not every little difficulty is worth a language feature. In the cases of several of the features adopted recently, a quick inspection has initially led me to believe that the feature was not interesting but study showed that it was. I just don't think you can delegate this kind of study to a group that is mostly inexperienced even at programming much less language design. I want my BDFL. As Davy Crockett said, "Be sure you're right, then go ahead." Paul Dubois

20 41

Unicode filenames
by Just van Rossum 11 Feb '03

11 Feb '03

MacOSX fully supports unicode filenames (utf-8 is used throughout), and I'm tempted to set Py_FileSystemDefaultEncoding to "utf8" for OSX. Jack pointed me to a long thread about unicode filenames that took place on python-dev last year, but I can't deduce from it whether there are any disadvantages of setting Py_FileSystemDefaultEncoding. Setting it seems to work wonderful. However, I'm a bit surprised that os.listdir() doesn't return unicode strings. Is that because it would break too much code? BTW. if I try to create a file with an 8-bit filename which is _not_ valid utf-8, I get a strange error: >>> f = open("\xff", "w") Traceback (most recent call last): File "<stdin>", line 1, in ? IOError: invalid mode: w >>> This exception is thrown when errno is EINVAL, which apparently can also mean that the filename arg is bad. Not sure if we can fix this. Just

4 7

Adding Japanese Codecs to the distro
by M.-A. Lemburg 11 Feb '03

11 Feb '03

Hisao SUZUKI has just recently uploaded a patch to SF which includes codecs for the Japanese encodings EUC-JP, Shift_JIS and ISO-2022-JP and wants to contribute the code to the PSF. The advantage of his codecs over the ones written by Tamito KAJIYAMA (http://www.asahi-net.or.jp/~rd6t-kjym/python/) lies in the fact that Hisao's codecs are small (88kB) and written in pure Python. This makes it much easier to adapt the codecs to special needs or to correct errors. Provided Hisao volunteers to maintain these codecs, I'd like to suggest adding them to Python's encodings package and making them the default implementations for the above encodings. Ideal would be if we could get Hisao and Tamito to team up to support these codecs (I put him on CC). Adding the codecs to the distribution would give Python a very good argument in the Japanese world and also help people working with XML or HTML targetting these locales. Thoughts ? -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/

10 37

thunks
by Guido van Rossum 11 Feb '03

11 Feb '03

I still don't have time to read this thread :-(, but I had a new idea that I'd like to fly here. If we want to support thunks that blend into the scope of their environment as well as thunks that introduce a new local scope, here's one possible way to do it: Thunk blending in: [<variable> =] <expression>: <block> Thunk introducing a new local scope: [<variable> =] <expression> lambda: <block> (Never mind the ugly things I've said about lambda in the past. :-) I'm confident that the parser can distinguish these, and I'm confident that the scope blending can be implemented using nested-scope cells. Adding formal parameters to the second case would go like this: [<variable> =] <expression> lambda <arguments>: <block> One concern I have in both cases: I think there are use cases for wanting more than one block. Glyph's example wants to define various callbacks for a remote call, and this would need multiple blocks. I think the syntax for this would have to use some kind of keyword-based continuation, e.g. [<variable> =] <expression> lambda: <block> and <expression> lambda: <block> ... Disclaimer: this is a half-baked idea. --Guido van Rossum (home page: http://www.python.org/~guido/)

29 74

bitwise operators and lambdas
by Jack Diederich 10 Feb '03

10 Feb '03

I don't expect action or consensus on these, but I thought I get them on record by posting 1) move bitwise operators to a package and deprecate them from the language as special syntax 2) use pipes as shorthand for lambdas re: 1) bitwise operators bitwise OR is used on 61 lines in Lib/ out of 70k+ lines. There is nothing inheritly un-pythonic about bitwise operators but they don't really come up much in a high level language like python. They just seem 'extra' esp since we spell || as 'or' and && as 'and'. sets are about the only thing to usefully define __xor__ and its kind. re: 2) lambda shorthand Most people hate lambdas, but I like em. They would beat listcomps in readability if it weren't for the clunky appearance of the string 'lambda' everywhere they are used. People seem to like list comps over filter/map mainly because they save keystrokes. old shorthand lambda:foo() ||foo() lambda x:foo(x) |x|foo(x) l = [x.guy for (x) in blah] l = map(|x|x.guy, blah) l = [x for (x) in blah if x < 7] l = filter(|x|x < 7, blah) l = [x.guy for (x) in blah if x.guy < 7] l = map(|x|x.guy, filter(|x|x.guy < 7, blah)) I also like the passing-something-on implication of using a pipe character. Plus I like the left-to-right reading of map/filter over listcomps. dreaming-of-pytohn-3k-ly, -jackdied

3 2

UNREF invalid object
by Gustavo Niemeyer 10 Feb '03

10 Feb '03

Hello! I've been playing with the bz2module to remove the file inheritance, as requested. While doing so, I've got the "UNREF invalid object" error when trying to run PyObject_Del(), inside a dealloc function, on an object which is not "externally" accessible (it's part of the class structure only, and has obj->ob_refcnt == 1, as expected). I'll replace the PyObject_Del() by Py_DECREF() which fixes the problem and feels more elegant to me anyway. OTOH, I wouldn't like to get over the problem without knowing what's really happening. Is this a common problem pattern? Thanks! -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ]

2 2