From pedroni@inf.ethz.ch Mon Apr 1 00:30:02 2002 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Mon, 1 Apr 2002 02:30:02 +0200 Subject: [Python-Dev] Evil isinstance() References: <02033119385806.11947@arthur> <02033121434808.11947@arthur> <200203312113.g2VLDmN23709@pcp742651pcs.reston01.va.comcast.net> <0203312329440J.11947@arthur> <200203312219.g2VMJt124097@pcp742651pcs.reston01.va.comcast.net> Message-ID: <003c01c1d914$68ab13a0$6d94fea9@newmexico> From: Guido van Rossum > > > This leaves me in the uncomfortable position that I don't know > > > what to recommend. :-( > > > > In your shoes, I'd recommend PEP 246 -- since, if I were in your > > shoes, I'd have approved it long ago. (We don't need to have > > interfaces as a distinct concept from types/classes to approve PEP > > 246 -- any types can serve as 'protocols' to make PEP 246 a > > reality). It would offer a very good solution to problems in this > > category, in my opinion. > > Maybe you can summarize it again using a different vocabulary? I find > that PEP very hard to read, but I recall liking your informal > explanation of it. Unfortunately I don't recall enough of that > explanation to go ahead and approve the PEP (I have no idea what > effect that would have). It's a tool, IMHO what is missing is a common dictionary of protocols and idioms, Alex says we don't need that, just let people have the tool and be inspired by other frameworks. E.g. how should one spell the example test: adapt(thearg,str) # but then what about unicode and should str have # an __adapt__ ? or adapt(thearg,StringLike) but then should every one define his StringLike, or should the std lib define one, should StringLike __conform__ use the +'' trick or not. Further the PEP suggests that a global registry for 3rd part adaptations would be useful (and give more degree-of-freedom to cope with the above issues) but it does not specify it, it specify just a global __import__-like hook for adaptation, but __import__ problems suggest that is not a valid solution... regards, Samuele Pedroni. From David Abrahams" <02033121434808.11947@arthur> <200203312113.g2VLDmN23709@pcp742651pcs.reston01.va.comcast.net> <0203312329440J.11947@arthur> <200203312219.g2VMJt124097@pcp742651pcs.reston01.va.comcast.net> <003c01c1d914$68ab13a0$6d94fea9@newmexico> Message-ID: <055701c1d917$d6420290$0202a8c0@boostconsulting.com> ----- Original Message ----- From: "Samuele Pedroni" > > Maybe you can summarize it again using a different vocabulary? I find > > that PEP very hard to read, but I recall liking your informal > > explanation of it. Unfortunately I don't recall enough of that > > explanation to go ahead and approve the PEP (I have no idea what > > effect that would have). > > It's a tool, IMHO what is missing is a common dictionary of protocols > and idioms, Agreed, establishing some useful protocols would really help, especially if these protocols correspond to objects provided by the core language and standard library. Warning: we might feel compelled to try to define what constitutes a Sequence, Number, or Mapping (not a bad idea anyway, IMO). > Alex says we don't need that, just let people > have the tool and be inspired by other frameworks. I don't think we'd need to try to be exhaustive, but IMO establishing some examples of good practice is important. It's also important to define the protocols that everyone will define over and over again if we don't do it (ahem: Sequence, Number, Mapping...) > E.g. how should one spell the example test: > > adapt(thearg,str) > # but then what about unicode and should str have > # an __adapt__ ? > > or > > adapt(thearg,StringLike) > > but then should every one define his StringLike, > or should the std lib define one, should StringLike > __conform__ use the +'' trick or not. > > Further the PEP suggests that a global > registry for 3rd part adaptations would be useful > (and give more degree-of-freedom to cope > with the above issues) > but it does not specify it, it specify just a global > __import__-like hook for adaptation, > but __import__ problems suggest that is not > a valid solution... >From my experience with the traits idiom in C++, I guess that some non-intrusive way to associate protocols and types will be very important. FWIW, I'm interested in this partly as an avenue to multimethod support for Python. -Dave From tim.one@comcast.net Mon Apr 1 02:26:10 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 31 Mar 2002 21:26:10 -0500 Subject: [Python-Dev] Re: PEP 279 In-Reply-To: <002901c1d7f2$8b04f560$2eb53bd0@othello> Message-ID: [Tim] >> Bingo. If you want coroutines, design a coroutine facility. >> Piling more semantics on to "yield" takes a delightfully simple yet >> powerful gimmick and hides it under ten kinds of obscurity. [Raymond Hettinger] > I don't see the obscurity. I know . As I said earlier in the msg, "generator exceptions do less damage (IMO) to simplicity than some of the other extensions". I had in mind above *all* the other gimmicks suggested to be layered on top of yield, including burying yield in what otherwise looked like listcomps, and the "generator parameter passing" idea. > This is the kind of tool that is invisible and out of the way until the > one day you really need it and the tool is right there in your hand (as > if by time machine). > > Throw wouldn't be used 98% of the time. I'd call it 99+%, but that was my original point: it's rarely needed. The (very) few times I've needed it, I was happy rolling my own. > ... > I didn't come up with these ideas out of the ether. It came up in a > real world application for producing index prints from jpegs. The question is then whether simple generators are a fork that hasn't yet grown enough tines, or whether your app really wanted a knife. That is, if you try to use a feature and find it's not enough for a specific problem, it doesn't follow that the feature is flawed. BTW, the cleanest way I can think of to implement generator exceptions would be to add a new opcode to the Python virtual machine, and that's a pretty heavy change for a rarely-needed minor convenience. > That being said, I accept that generator parameter passing and exception > passing are doomed. Heh. I started pushing for generators in the very early 90's. The idea was rejected multiple times, but usually because somebody else hijacked the idea in favor of continuations instead. Pare away the over-elaborate, and something simple that remains may have a chance, if you just keep poking at it for a decade . > Here is my alternate proposal that is much simpler and doesn't encourage > weirdness. > [Developers feel free to stomp on the idea but please don't smack > me around for thinking it up]. > > Specification for Generator Attributes: > > 1. Allow attributes to be assigned to generator objects. That's doable. You can already assign attributes to generator-functions, so I figure this means generator-iterators (to which you cannot currently assign attributes). > 2. Provide a means of accessing those attributes from within the > generator by using a reference to __self__. < > def mygen(): > print __self__.greeting > yield 'hello' > print __self__.greeting > > g = mygen() > g.greeting = 'Good morning' > print g.next() > g.greeting = 'Good night' This isn't an, umm, compelling example. Why would I *really* want to do this? If I really wanted to, I expect I'd again make the generator a method of a class and use vanilla instance variables -- or maybe I would really want coroutines. > Advantages: Uses the standard python attribute assignment idiom > so there is no learning curve and no surprises. Ditto for instance variables. > Easily implementable without new keywords, new builtins, or parser > magic. That isn't so: the parser is going to have to recognize "__self__" as a special token, and generate special code to make it resolve to the generator-iterator object derived from the generator-function. This isn't at all like the current use of "self" in methods, because in that context "self" is just a named parameter like any other, and lives in the function's local namespace like any other named parameter. This new gimmick would make more sense if the generator's 'def' were spelled def mygen(__self__): but then it gets uglier if mygen is also a method. > Provides a data sharing solution that avoids the yield / next() matching > problem, avoids enclosing classes, and avoid global variables. Simple > and neat. I don't think it's so simple, as there's nothing else like it in Python now: it can't really be explained in terms of anything that already exists. Names always live in the local, global or builtin namespaces. __self__ would have to live in the local namespace to make any sense, but *no* name shows up "by magic" in a local namespace now. > Disads: Introduces a new system variable, __self__. The rub is that it's a system variable unlike any other. You may find it's more convenient to pass an explicit "message carrier" to mygen: class Carrier: pass # just a dummy to hold names def mygen(__self__): __self__.this and __self__.that, exactly as above msgs = Carrier() g = mygen(msgs) msgs.greeting = 'Good morning' print g.next() etc. If this is actually a common pattern for you, a very simple wrapper class could capture it, so that you could write g = Wrapper(mygen) g.greeting = 'Good morning') etc. > P.S. No one needs to say anything mean to make me go away. Only Guido wants you to go away , and he's never mean, just Dutch. I don't know that generators will never be enhanced, but they're a brand-new feature to the vast bulk of Python programmers, and it would be prudent to wait a few years to see which issues come up repeatedly and don't find a reasonably simple solution via combining them with other Python features (like classes indeed: "doesn't need a class" isn't really a selling point). It took 'em a decade to get in, and they're already more powerful than in the languages Python borrowed them from (primarily Icon and CLU, with a dash of Sather). From tim.one@comcast.net Mon Apr 1 02:50:32 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 31 Mar 2002 21:50:32 -0500 Subject: [Python-Dev] Re: PEP 279 In-Reply-To: <003601c1d7f4$33139d00$2eb53bd0@othello> Message-ID: [Raymond Hettinger] > Okay, here's what we have so far: > > iterindexed()-- five syllables is a mouthfull > index() -- nice verb but could be confused the .index() method > indexed() -- widely liked however adjectives should be avoided > count() -- direct and explicit but often used in other contexts > itercount() -- direct, explicit and hated by more than one person > enumerate() -- a contender but doesn't mention iteration or indices > iteritems() -- already used by dictionaries for key:value pairs Since Guido is sticking to bool() for his current PEP, I guess that leaves truth() open for this . iotazip() is a natural for those with both APL and Haskell backgrounds, and sounds way cool if you're burned out. ordinalize() is an even clumsier way to say enumerate(). poll() would confuse everyone almost equally. countoff() brings to mind a common isomorphic procedure in American schools ("One!", "Two!", "Three!", ...). I kinda like it, although zerobasedcountoff() may be more appropriate here . rollcall() is cute. paginate() is descriptive for one specific application. If indexed() is disliked just because it's an adjective, Google finds only one hit for indicify(). indexify() gets 103. I'm amazed that Marc-Andre's irange() isn't already in the list (it's part of his mxTools Python extension). From aahz@pythoncraft.com Mon Apr 1 02:58:23 2002 From: aahz@pythoncraft.com (Aahz) Date: Sun, 31 Mar 2002 21:58:23 -0500 Subject: [Python-Dev] Namespaces In-Reply-To: References: <002901c1d7f2$8b04f560$2eb53bd0@othello> Message-ID: <20020401025823.GA29354@panix.com> On Sun, Mar 31, 2002, Tim Peters wrote: > > Names always live in the local, global or builtin namespaces. Well, no. First-level names, yes, but each object itself is a namespace, so every name will be bound to an object that can contain names. To use my favorite example: def f(): pass f.permissions = 'author' (I'm being picky about this because I'm in the middle of writing my PyPerl tutorial for OSCON, and I think this is one of the areas that really needs to be accurate -- so if I'm wrong, I want to know about it.) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ Why is this newsgroup different from all other newsgroups? From fdrake@acm.org Mon Apr 1 03:03:58 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Sun, 31 Mar 2002 22:03:58 -0500 Subject: [Python-Dev] Re: PEP 279 In-Reply-To: References: <003601c1d7f4$33139d00$2eb53bd0@othello> Message-ID: <15527.52766.458478.619424@grendel.zope.com> [Raymond Hettinger] > enumerate() -- a contender but doesn't mention iteration or indices I certainly like this name the best of the options I've heard so far. It seems clear enough to me, at any rate. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From tim.one@comcast.net Mon Apr 1 03:52:13 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 31 Mar 2002 22:52:13 -0500 Subject: [Python-Dev] pymalloc killer In-Reply-To: <200203300110.g2U1Arm17301@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > On the other hand, the less sophisticated extension writers never > release the GIL at all. That's an excellent point! I don't know how much to bank it, but it's semi-reassuring. > All I can do is encourage you to keep trying. I find it quite > encouraging that you found a constant-time test for non-arena memory > at all. David Abrahams had the key insight there. Those C++ guys really do have some good ideas, if you can whittle away all the baffling syntax to get at the pure tiny thoughts they're thinking . > Making it thread-safe should only add a tiny constant time. It does now, although it's balancing on a nanometer pinhead. That's good enough so long as nobody sneezes in the general direction of the code. From DavidA@ActiveState.com Mon Apr 1 05:20:48 2002 From: DavidA@ActiveState.com (David Ascher) Date: Sun, 31 Mar 2002 21:20:48 -0800 Subject: [Python-Dev] Re: PEP 279 References: Message-ID: <3CA7EE30.55A52@ActiveState.com> My current fave is 'indexer'. Something which creates an index. --da From optin_travel@email.it Mon Apr 1 06:31:18 2002 From: optin_travel@email.it (optin_travel@email.it) Date: Mon, 1 Apr 2002 01:31:18 -0500 Subject: [Python-Dev] (no subject) Message-ID: This is a multi-part message in MIME format. ------_NextPart_9651631181 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: QUOTED-PRINTABLE 3 Luxury Vacations - 199.00 Orlando - Bahamas - Cancun Mexico! This is not a timeshare! No property tours! No Catch! 1-800-922-1432 CALL NOW TOLL FREE Operators are standing by. Cruise - Dive - Snorkel - Fine Dining Visit - Disney World - Sea World - Universal ! 100% Transferable - One year to travel ! ------_NextPart_9651631181 Content-Type: text/html; charset="ISO-8859-1" Content-Transfer-Encoding: QUOTED-PRINTABLE 3 Luxury Vacations!
------_NextPart_9651631181-- From optin_travel@email.it Mon Apr 1 07:15:19 2002 From: optin_travel@email.it (optin_travel@email.it) Date: Mon, 1 Apr 2002 02:15:19 -0500 Subject: [Python-Dev] (no subject) Message-ID: This is a multi-part message in MIME format. ------_NextPart_3613094360 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: QUOTED-PRINTABLE 3 Luxury Vacations 199.00 Orlando, Bahamas, Cancun This is not a timeshare. No property tours! No Catch! 1-800-922-1432 TOLL FREE - CALL NOW Operators are standing by. Cruise the Bahamas - Cancun - Dive - Snorkel - Sun - Fun ! Visit Disney World ! See Mickey ! Ride the Coasters - Sea World ! ------_NextPart_3613094360 Content-Type: text/html; charset="ISO-8859-1" Content-Transfer-Encoding: QUOTED-PRINTABLE 3 Luxury Vacations!
------_NextPart_3613094360-- From martin@v.loewis.de Mon Apr 1 07:47:55 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 01 Apr 2002 09:47:55 +0200 Subject: [Python-Dev] Moving forward on the object memory API In-Reply-To: References: Message-ID: Tim Peters writes: > [martin@v.loewis.de] > > The third part is easy to answer. If it is deprecated, and used in the > > core or an extension, that use must be removed. > > This is too easy of an answer, as the docs *already* make such distinctions. > Read the tail end of the PyMem_ docs (PyMem_UPPERCASE is supposedly already > deprecated, but only in extension modules). There is something wrong here, then. The author of this documentation clearly meant that they are *internal*, and thus should have been called _PyMem_UPPERCASE. I'd be in favour of renaming them, thus actively breaking extension modules that use them at compile time. Regards, Martin From martin@v.loewis.de Mon Apr 1 08:00:04 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 01 Apr 2002 10:00:04 +0200 Subject: [Python-Dev] Namespaces In-Reply-To: <20020401025823.GA29354@panix.com> References: <002901c1d7f2$8b04f560$2eb53bd0@othello> <20020401025823.GA29354@panix.com> Message-ID: Aahz writes: > Well, no. First-level names, yes, but each object itself is a > namespace, so every name will be bound to an object that can contain > names. To use my favorite example: > > def f(): > pass > > f.permissions = 'author' Except that those are not names, they are attributes. Namespaces nest, in Python; permissions does not live in namespace, which becomes obvious if you try to nest it def f(): return permissions f.permissions = 'author' print f() Traceback (most recent call last): File "a.py", line 5, in ? print f() File "a.py", line 2, in f return permissions NameError: global name 'permissions' is not defined Regards, Martin From gsw@agere.com Mon Apr 1 13:55:13 2002 From: gsw@agere.com (Gerald S. Williams) Date: Mon, 1 Apr 2002 08:55:13 -0500 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: Message-ID: Guido van Rossum wrote: > I offer the following PEP for review by the community. If it receives > a favorable response, it will be implemented in Python 2.3. I like almost all of it as written. > 2) Should str(True) return "True" or "1": "1" might reduce > backwards compatibility problems, but looks strange to me. I also prefer "True". int(True) returns 1 if you need it. > 3) Should the constants be called 'True' and 'False' I prefer "True" here also (this is Python, not C). > 4) Should we strive to eliminate non-Boolean operations on bools > in the future, through suitable warnings, so that e.g. True+1 > would eventually (e.g. in Python 3000 be illegal). Personally, > I think we shouldn't; 28+isleap(y) seems totally reasonable to > me. No need. It is only equivalence operations that cause problems. > It has been suggested that, in order to satisfy user expectations, > for every x that is considered true in a Boolean context, the > expression x == True should be true, and likewise if x is > considered false, x == False should be true. This is of course > impossible; it would mean that e.g. 6 == True and 7 == True, from > which one could infer 6 == 7. Similarly, [] == False == None > would be true, and one could infer [] == None, which is not the > case. I'm not sure where this suggestion came from; it was made > several times during the first review period. For truth testing > of a value, one should use "if", e.g. "if x: print 'Yes'", not > comparison to a truth value; "if x == True: print 'Yes'" is not > only wrong, it is also strangely redundant. I (was at least one of the people who) brought this up. Without getting tied up in the discussion about which should inherit from which: in boolean context, 6 == 7 and [] == None. This does not imply that they are equivalent elsewhere. The problem is the equivalence test itself. It may be true that "if x == True:" is redundant, in which case you may want to disallow such comparisons. But performing an integer comparison will confuse perfectly rational people at least some of the time. Consider that the main source of confusion won't come from comparisons to True, but comparisons to False. These two statements might look identical to the casual observer, but they really aren't: if not expr: if expr == False: I think *that* is probably the best way to summarize my concern. -Jerry -O Gerald S. Williams, 22Y-103GA : mailto:gsw@agere.com O- -O AGERE SYSTEMS, 555 UNION BLVD : office:610-712-8661 O- -O ALLENTOWN, PA, USA 18109-3286 : mobile:908-672-7592 O- From tim.one@comcast.net Mon Apr 1 14:41:57 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 01 Apr 2002 09:41:57 -0500 Subject: [Python-Dev] Moving forward on the object memory API In-Reply-To: Message-ID: [Tim] >> Read the tail end of the PyMem_ docs (PyMem_UPPERCASE is >> supposedly already deprecated, but only in extension modules). [martin@v.loewis.de] > There is something wrong here, then. That's for sure . > The author of this documentation clearly meant that they are *internal*, > and thus should have been called _PyMem_UPPERCASE. I doubt the author meant that: in 1.5.2, PyMem_DEL was the only "legal" way to free object memory, and even if people didn't know that from reading the docs, they picked it up via copying Python's Objects/xxmodule.c (which used PyMem_DEL through revision 2.15). > I'd be in favour of renaming them, thus actively breaking extension > modules that use them at compile time. That would break every extension type written for 1.5.2; the relevant ones from Neil's random sampling are: DB2-Python-0.991 Allocates with PyObject_NEW and deallocates with PyMem_DEL DCOracle-1.3.1b1 Allocates with PyObject_NEW and deallocates with PyMem_DEL Kinterbasdb Allocates with PyObject_NEW and deallocates with PyMem_DEL PyGreSQL-3.3-pre011203 Allocates with PyObject_NEW and deallocates with PyMem_DEL avl-2.0 Allocates with PyObject_NEW and deallocates with PyMem_DEL sybase-0.33 Allocates with PyObject_NEW and deallocates with PyMem_DEL pysdl-0.0.7 Allocates with PyObject_NEW and deallocates with PyMem_DEL pyshout-0.0.6 Allocates with PyObject_NEW and deallocates with PyMem_DEL pwin Allocates with PyObject_NEW and deallocates with PyMem_DEL I'm all in favor of deprecating PyMem_DEL, so long as deprecation doesn't mean anything . An alternative-- which no longer poses an implementation problem --is to say that from 2.3 onward, PyMem_Del, PyMem_DEL, PyObject_Del, PyObject_DEL, PyObject_FREE, and PyObject_Free, are all exactly the same thing (as has always been true in reality, except when pymalloc was enabled in 2.2 or before). At least 4 of those spellings should be deprecated then, but gently. Whether PyMem_Free and PyMem_FREE also belong in that pile isn't clear. From nas@python.ca Mon Apr 1 14:54:57 2002 From: nas@python.ca (Neil Schemenauer) Date: Mon, 1 Apr 2002 06:54:57 -0800 Subject: [Python-Dev] Moving forward on the object memory API In-Reply-To: <200203312107.g2VL7xP23687@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Sun, Mar 31, 2002 at 04:07:59PM -0500 References: <20020331110813.A22515@glacier.arctrix.com> <200203312107.g2VL7xP23687@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020401065457.A24937@glacier.arctrix.com> Guido van Rossum wrote: > > Next, we base PyObject_{MALLOC,REALLOC,FREE} on pymalloc. Basically: > > > > #ifdef WITH_PYMALLOC > > #define PyObject_MALLOC(n) _PyMalloc_Malloc(n) > > #define PyObject_REALLOC(op, n) _PyMalloc_Realloc((void *)(op), (n)) > > #define PyObject_FREE(op) _PyMalloc_Free((void *)(op)) > > #else > > #define PyObject_MALLOC(n) PyMem_MALLOC(n) > > #define PyObject_REALLOC(op, n) PyMem_REALLOC((void *)(op), (n)) > > #define PyObject_FREE(op) PyMem_FREE((void *)(op)) > > #endif > > Couldn't these always use the first set of definitions? I suppose > that when configured --without-pymalloc, _PyMalloc_Malloc and friends > are aliases for malloc? No, the _PyMalloc_* functions are only available if pymalloc is enabled. We could make it so that the _PyMalloc_* functions are always available. People who don't use pymalloc would take a performance hit though. > Also, do we need these at all? (Hm, I must be missing something, but > I can't figure what.) Do you mean PyObject_{MALLOC,REALLOC,FREE}? I think we do. There needs to be a way to allocate memory using the object allocator. I like PyObject_MALLOC better than PyMalloc_MALLOC and it was also present in previous releases. > > PyMem_DEL and PyMem_Free need to call pymalloc: > > > > #define PyMem_DEL(op) PyObject_FREE(op) > > Why not _PyMalloc_Malloc? See above (pymalloc may be disabled). > > We go through the source and replace all the PyMem_* function calls with > > the equivalent PyMem_* macro calls and replace PyMem_DEL with > > PyMem_FREE. > > Or PyObject_Del (depending on the situation)? Nope. There should be no place in the CVS source that calls PyMem_DEL when it should be PyObject_Del. > Is there a difference between PyMem_MALLOC and PyMem_NEW? Between > PyMem_REALLOC and PyMem_RESIZE? If there isn't, shouldn't we > recommend one set and deprecate the other? I think I like the NEW > family best (it's got seniority). NEW and RESIZE are type based while MALLOC and REALLOC work by bytes. I think you need them both. > I suppose the patches that changed PyObject_New into PyMalloc_New can > be reverted now? Yes. Neil From nas@python.ca Mon Apr 1 14:56:38 2002 From: nas@python.ca (Neil Schemenauer) Date: Mon, 1 Apr 2002 06:56:38 -0800 Subject: [Python-Dev] Moving forward on the object memory API In-Reply-To: <200203312227.g2VMRNh24358@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Sun, Mar 31, 2002 at 05:27:23PM -0500 References: <20020331110813.A22515@glacier.arctrix.com> <20020331141600.B23023@glacier.arctrix.com> <200203312227.g2VMRNh24358@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020401065638.B24937@glacier.arctrix.com> Guido van Rossum wrote: > No, it's for the binary API. There's a test in Py_InitModule() that > checks whether a dynamically loaded extension uses the same API as the > core, and that's definitely a binary compatibility issue. I was under the impression that the binary API changes in every minor release. Neil From martin@v.loewis.de Mon Apr 1 15:19:25 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 01 Apr 2002 17:19:25 +0200 Subject: [Python-Dev] Moving forward on the object memory API In-Reply-To: References: Message-ID: Tim Peters writes: > > I'd be in favour of renaming them, thus actively breaking extension > > modules that use them at compile time. > > That would break every extension type written for 1.5.2; the relevant ones > from Neil's random sampling are: There are different levels of breakage, though: low: generates a warning when invoked middle: fails to compile high: compiles, but corrupts memory at run-time If even the middle level is too much breakage, I'd be in favour of deprecating things in the low level: map them to a function that produces a warning when invoked. Regards, Martin From tim.one@comcast.net Mon Apr 1 15:23:34 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 01 Apr 2002 10:23:34 -0500 Subject: [Python-Dev] Moving forward on the object memory API In-Reply-To: <20020401065638.B24937@glacier.arctrix.com> Message-ID: [Neil Schemenauer] > I was under the impression that the binary API changes in every minor > release. In recent reality it has, so you young sprouts can be forgiven for not realizing it's not mandatory to break binary compatibility across releases. The longevity record for binary compatibility extended from 18-Jan-1997 to 14-Mar-2000; note that 1.5.2 was released during that stretch, on 13-Apr-1999. The full history is in modsupport.h, under PYTHON_API_STRING. From guido@python.org Mon Apr 1 15:33:08 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 01 Apr 2002 10:33:08 -0500 Subject: [Python-Dev] Moving forward on the object memory API In-Reply-To: Your message of "Mon, 01 Apr 2002 06:56:38 PST." <20020401065638.B24937@glacier.arctrix.com> References: <20020331110813.A22515@glacier.arctrix.com> <20020331141600.B23023@glacier.arctrix.com> <200203312227.g2VMRNh24358@pcp742651pcs.reston01.va.comcast.net> <20020401065638.B24937@glacier.arctrix.com> Message-ID: <200204011533.g31FX8D32577@pcp742651pcs.reston01.va.comcast.net> > Guido van Rossum wrote: > > No, it's for the binary API. There's a test in Py_InitModule() that > > checks whether a dynamically loaded extension uses the same API as the > > core, and that's definitely a binary compatibility issue. > > I was under the impression that the binary API changes in every minor > release. The API version number doesn't always. Maybe we're going about this wrong though: when we only *add* to the API, we don't change the API version number, on the theory that extensions that worked before still work. But of course extensions that use the added API functions won't work on the older systems, despite having the same API number. (However, they will get a different kind of error, from the linker.) I think I try to change the API version number when there are incompatible changes in object lay-out which would trip you up even if the functions haven't changed. An example would be adding the hash field to the string object, which requires recompilation for every extension that uses the PyString_AS_STRING macro. (This happened many years ago of course.) --Guido van Rossum (home page: http://www.python.org/~guido/) From martin@v.loewis.de Mon Apr 1 15:25:35 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 01 Apr 2002 17:25:35 +0200 Subject: [Python-Dev] Moving forward on the object memory API In-Reply-To: <20020401065638.B24937@glacier.arctrix.com> References: <20020331110813.A22515@glacier.arctrix.com> <20020331141600.B23023@glacier.arctrix.com> <200203312227.g2VMRNh24358@pcp742651pcs.reston01.va.comcast.net> <20020401065638.B24937@glacier.arctrix.com> Message-ID: Neil Schemenauer writes: > > No, it's for the binary API. There's a test in Py_InitModule() that > > checks whether a dynamically loaded extension uses the same API as the > > core, and that's definitely a binary compatibility issue. > > I was under the impression that the binary API changes in every minor > release. (By minor, you mean the y in x.y.z?) On Windows, yes - the DLL name changes, causing problems for old extensions. On Unix, not necessarily - atleast not in incompatible ways. I think the API didn't change between 1.6 and 2.0 (see Include/modsupport.h); I'm not sure whether 2.3 will see an API change. Regards, Martin From tanzer@swing.co.at Mon Apr 1 07:14:59 2002 From: tanzer@swing.co.at (Christian Tanzer) Date: Mon, 01 Apr 2002 09:14:59 +0200 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: Your message of "Sat, 30 Mar 2002 04:35:32 EST." Message-ID: Tim Peters wrote: > [Christian Tanzer] > > ... > > If str and repr were to behave differently from each other, I'd expec= t > > repr(True) to return "1" and str(True) to return "True". Interchangin= g > > that seems strange. > > We try to ensure that eval(repr(x)) reproduce x whenever reasonably > possible. That best way to do that here is for repr(True) to return > 'True'. I understood the motivation for `repr(bool(1)) =3D=3D "True"`. What I wanted to say is that having repr return a readable result and str return a result looking like an internal representation runs counter to the usual conventions used for repr and str. > > OTOH, I'm worried about backwards compatibility. > > That's the only reason for str(True) to return '1'. In this case, I'd value consistency between repr/str higher than backwards compatibility. > How about str(True) return '1' but str(False) return 'False'? That's > what a committee would compromise on . Any self respecting committee would try harder and probably come up with a really symmetric compromise: value str repr True '1' 'True' False 'False' '0' <1.4. wink> > > How will bool influence '__nonzero__'? Currently, Python insists that= > > '__nonzero__ should return an int'. Will this be changed to 'should > > return a bool'? > > I hope so. > > > As that would break existing code, I hope not. > > No, "should" !=3D "must". Fine display of sophisticated hair splitting, but: >>> class Foo: ... def __nonzero__(self): return None ... >>> f =3D Foo () >>> not f Traceback (most recent call last): File "", line 1, in ? TypeError: __nonzero__ should return an int >>> I don't care about `should` vs. `must` as long as the new and improved Python won't spit an exception at __nonzero__ code returning an int instead of a bool. > Guido won't call code that returns an int here broken, although he > might call it deprecated, and leave you wondering when the hammer will > drop . Nice tease. But Guido's hammer with regard to future strictification of bool is exactly what makes me uneasy. -- = Christian Tanzer tanzer@swing.co.= at Glasauergasse 32 Tel: +43 1 876 62 = 36 A-1130 Vienna, Austria Fax: +43 1 877 66 = 92 From martin@v.loewis.de Mon Apr 1 16:01:08 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 01 Apr 2002 18:01:08 +0200 Subject: [Python-Dev] Moving forward on the object memory API In-Reply-To: <200204011533.g31FX8D32577@pcp742651pcs.reston01.va.comcast.net> References: <20020331110813.A22515@glacier.arctrix.com> <20020331141600.B23023@glacier.arctrix.com> <200203312227.g2VMRNh24358@pcp742651pcs.reston01.va.comcast.net> <20020401065638.B24937@glacier.arctrix.com> <200204011533.g31FX8D32577@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > I think I try to change the API version number when there are > incompatible changes in object lay-out which would trip you up even if > the functions haven't changed. An example would be adding the hash > field to the string object, which requires recompilation for every > extension that uses the PyString_AS_STRING macro. Or removal of the ob_size field from the cell object... Regards, Martin From fdrake@acm.org Mon Apr 1 16:37:06 2002 From: fdrake@acm.org (Fred L. Drake) Date: Mon, 1 Apr 2002 11:37:06 -0500 (EST) Subject: [Python-Dev] [development doc updates] Message-ID: <20020401163706.D36FF18ED64@grendel.zope.com> The development version of the documentation has been updated: http://python.sourceforge.net/maint-docs/ Documentation being prepared for Python 2.2.1. This version includes Andrew Kuchling's "What's New in Python 2.2" alongside the original documents. From gsw@agere.com Mon Apr 1 17:04:38 2002 From: gsw@agere.com (Gerald S. Williams) Date: Mon, 1 Apr 2002 12:04:38 -0500 Subject: [Python-Dev] Re: PEP 285: Adding a bool typen In-Reply-To: Message-ID: I'm not questioning whether it's broken or not, just stating that this is a likely mistake for people to make. Especially since the latter case does work if expr is the result of an operation with a logical result but not for other expressions that are often used in a logical context. I probably should have used "expr_result" rather than "expr". I really meant for "expr" to be a variable, and was thinking about the case where you would do this: expr = ...some expression with a "logical" outcome... if not expr: .... In a strongly-typed language, this isn't an issue, since you'd have declared "expr" as a boolean type. In that case, "not expr" is the same as "expr == False". Considering Python's roots as a teaching language, it would be preferable to immediately catch situations where Boolean values are compared to non-Boolean ones and do *something* about it. -Jerry -O Gerald S. Williams, 22Y-103GA : mailto:gsw@agere.com O- -O AGERE SYSTEMS, 555 UNION BLVD : office:610-712-8661 O- -O ALLENTOWN, PA, USA 18109-3286 : mobile:908-672-7592 O- Paul Svensson wrote: > On Mon, 1 Apr 2002, Gerald S. Williams wrote: > >Consider that the main source of confusion won't come from > >comparisons to True, but comparisons to False. These two > >statements might look identical to the casual observer, but > >they really aren't: > > > > if not expr: > > > > if expr == False: > > Of course they aren't; > the first example is good code, > the second is broken, > and deserves to lose. From aahz@pythoncraft.com Mon Apr 1 18:56:57 2002 From: aahz@pythoncraft.com (Aahz) Date: Mon, 1 Apr 2002 13:56:57 -0500 Subject: [Python-Dev] Namespaces In-Reply-To: References: <002901c1d7f2$8b04f560$2eb53bd0@othello> <20020401025823.GA29354@panix.com> Message-ID: <20020401185657.GA6094@panix.com> On Mon, Apr 01, 2002, Martin v. Loewis wrote: > Aahz writes: >> >> Well, no. First-level names, yes, but each object itself is a >> namespace, so every name will be bound to an object that can contain >> names. To use my favorite example: >> >> def f(): >> pass >> >> f.permissions = 'author' > > Except that those are not names, they are attributes. Namespaces > nest, in Python; permissions does not live in namespace, which becomes > obvious if you try to nest it > > def f(): > return permissions > > f.permissions = 'author' > print f() > > Traceback (most recent call last): > File "a.py", line 5, in ? > print f() > File "a.py", line 2, in f > return permissions > NameError: global name 'permissions' is not defined You're only correct if you define namespaces as being the set of nested first-level lookups. If you define "attribute" as "name bound to an object namespace", then we're pretty much in agreement. We certainly talk often enough of module namespaces; I think it makes sense to expand that usage to all objects. It seems pretty clear to me that f.permissions does not live in the local/global/builtins namespace, but it sure lives in f's namespace. Because attributes are names like any other name and can be bound to any Python object, I think it lends crucial orthogonality to say that attributes live in object namespace. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ Why is this newsgroup different from all other newsgroups? From fdrake@acm.org Mon Apr 1 20:33:00 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 1 Apr 2002 15:33:00 -0500 Subject: [Python-Dev] New home for development version of docs Message-ID: <15528.50172.390695.897521@grendel.zope.com> The "in-developement" version of the documentation, previously found at python.sourceforge.net, is now available from the development area on python.org: http://www.python.org/dev/doc/ -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From jeremy@zope.com Mon Apr 1 21:00:02 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 1 Apr 2002 16:00:02 -0500 Subject: [Python-Dev] Namespaces In-Reply-To: <20020401185657.GA6094@panix.com> References: <002901c1d7f2$8b04f560$2eb53bd0@othello> <20020401025823.GA29354@panix.com> <20020401185657.GA6094@panix.com> Message-ID: <15528.51794.642978.22967@slothrop.zope.com> >>>>> "A" == aahz writes: A> You're only correct if you define namespaces as being the set of A> nested first-level lookups. If you define "attribute" as "name A> bound to an object namespace", then we're pretty much in A> agreement. We certainly talk often enough of module namespaces; A> I think it makes sense to expand that usage to all objects. I think it's fairly common for people to talk about an object's attributes as its namespace. The language reference does not call this a namespace. In fact, the lang ref doesn't define namespace and often uses other words to discuss naming: names, blocks, scopes, environment. Perhaps namespace is too fuzzy and we should introduce a precise term to talk about the attribute "namespace" of an object. The lang ref does say that the namespace of a class definition becomes the attribute dictionary of the class. So "attribute dictionary" seems to be preferred to "object namespace" <0.5 wink>. A> It seems pretty clear to me that f.permissions does not live in A> the local/global/builtins namespace, but it sure lives in f's A> namespace. Because attributes are names like any other name and A> can be bound to any Python object, I think it lends crucial A> orthogonality to say that attributes live in object namespace. Some people will be confused that a function f has two namespaces -- the namespace for its code object and the namespace for its attributes. This reinforces my sense that we shouldn't use the term namespace when we are trying to be precise. (Or that we need a precise definition of namespace and a clear way to disambiguate the two kinds of namespaces assocaited with a function.) I'd also like to observe that the language reference doesn't say anything about how attributes of objects are resolved. The tutorial says a bit in the chapter on classes, but none of this made it to the language reference. It would be nice to flesh out these issues in the reference manual. Jeremy From guido@python.org Mon Apr 1 21:21:39 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 01 Apr 2002 16:21:39 -0500 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: Your message of "Mon, 01 Apr 2002 09:14:59 +0200." References: Message-ID: <200204012121.g31LLdk00743@pcp742651pcs.reston01.va.comcast.net> > Nice tease. But Guido's hammer with regard to future strictification > of bool is exactly what makes me uneasy. Can you elaborate that? I thought I made it pretty clear I didn't want to strictify bool (though I did open up the question as to whether other people want that). Is that what you call a hammer? --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@pythoncraft.com Mon Apr 1 21:22:32 2002 From: aahz@pythoncraft.com (Aahz) Date: Mon, 1 Apr 2002 16:22:32 -0500 Subject: [Python-Dev] Namespaces In-Reply-To: <15528.51794.642978.22967@slothrop.zope.com> References: <002901c1d7f2$8b04f560$2eb53bd0@othello> <20020401025823.GA29354@panix.com> <20020401185657.GA6094@panix.com> <15528.51794.642978.22967@slothrop.zope.com> Message-ID: <20020401212232.GA14508@panix.com> On Mon, Apr 01, 2002, Jeremy Hylton wrote: > > I think it's fairly common for people to talk about an object's > attributes as its namespace. The language reference does not call > this a namespace. In fact, the lang ref doesn't define namespace and > often uses other words to discuss naming: names, blocks, scopes, > environment. > > Perhaps namespace is too fuzzy and we should introduce a precise term > to talk about the attribute "namespace" of an object. Actually, namespace seems fine and clear to me; what IMO needs clarification is explicit namespaces (i.e. through object references) versus implicit/execution-based namespaces (builtin/global/local). Which reminds me of another annoyance in that "global" is strictly speaking "module global". > The lang ref does say that the namespace of a class definition becomes > the attribute dictionary of the class. So "attribute dictionary" > seems to be preferred to "object namespace" <0.5 wink>. Yeah, those sound synonymous to me, but four syllables beats seven when I'm teaching. ;-) > Aahz: > A> It seems pretty clear to me that f.permissions does not live in > A> the local/global/builtins namespace, but it sure lives in f's > A> namespace. Because attributes are names like any other name and > A> can be bound to any Python object, I think it lends crucial > A> orthogonality to say that attributes live in object namespace. > > Some people will be confused that a function f has two namespaces -- > the namespace for its code object and the namespace for its > attributes. This reinforces my sense that we shouldn't use the term > namespace when we are trying to be precise. (Or that we need a > precise definition of namespace and a clear way to disambiguate the > two kinds of namespaces assocaited with a function.) I'm fine with "local scope" and "object attributes" to disambiguate them; I just think it's important that people understand that a name is a name is a name, and all names live in *some* namespace. > I'd also like to observe that the language reference doesn't say > anything about how attributes of objects are resolved. The tutorial > says a bit in the chapter on classes, but none of this made it to the > language reference. > > It would be nice to flesh out these issues in the reference manual. Well, once we've got this hashed out, it's a SMOW. ;-) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ Why is this newsgroup different from all other newsgroups? From jeremy@alum.mit.edu Mon Apr 1 21:24:53 2002 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Mon, 01 Apr 2002 16:24:53 -0500 (EST) Subject: [Python-Dev] restrictions on global statement Message-ID: <0GTW003O5RHH6H@mtaout01.icomcast.net> The lang ref says: Names listed in a \keyword{global} statement must not be defined as formal parameters or in a \keyword{for} loop control target, \keyword{class} definition, function definition, or \keyword{import} statement. (The current implementation does not enforce the latter two restrictions, but programs should not abuse this freedom, as future implementations may enforce them or silently change the meaning of the program.) While this is all technically correct, the second paragraph is a bit misleading. The only case enforced by the compiler is that a formal parameter can't be declared global. Do we actually care about this rule? The lang ref would be shorter if it were silent on the issue, and it would more closely describe the way Python has behaved as long as I've been using it. Jeremy From jeremy@zope.com Mon Apr 1 21:59:45 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 1 Apr 2002 16:59:45 -0500 Subject: [Python-Dev] new section on naming and binding in reference manual Message-ID: <15528.55377.951848.352486@slothrop.zope.com> I replaced the old "execution frames" section of the reference manual with the naming rules from Appendix A.1 this afternoon. I made an attempt to be thorough about noting all the rules and corner cases that the old documentation covered, but I may have missed something. For example, I deleted the table of code blocks and namespaces. I think all the rules it described are explained in the text. (And I think most people are better off not thinking too much about the cases it describe <0.2 wink>.) I'd appreciate some review to make sure I've not omitted something that's important. Jeremy From David Abrahams" <20020401025823.GA29354@panix.com><20020401185657.GA6094@panix.com><15528.51794.642978.22967@slothrop.zope.com><07a301c1d9c4$1ee2ec10$0202a8c0@boostconsulting.com> <15528.54744.182533.107249@slothrop.zope.com> Message-ID: <07db01c1d9c8$de400a30$0202a8c0@boostconsulting.com> ----- Original Message ----- From: "Jeremy Hylton" > > The problem, which your response highlights, is what to call the thing > that isn't a namespace. I think '"attribute" things' suboptimal. How about "object"? Just about anything can have attributes. From fdrake@acm.org Mon Apr 1 22:06:24 2002 From: fdrake@acm.org (Fred L. Drake) Date: Mon, 1 Apr 2002 17:06:24 -0500 (EST) Subject: [Python-Dev] [development doc updates] Message-ID: <20020401220624.423C818EAD1@grendel.zope.com> The development version of the documentation has been updated: http://www.python.org/dev/doc/devel/ Updated documentation on lexical scoping in the language reference. From aahz@pythoncraft.com Mon Apr 1 22:11:43 2002 From: aahz@pythoncraft.com (Aahz) Date: Mon, 1 Apr 2002 17:11:43 -0500 Subject: [Python-Dev] new section on naming and binding in reference manual In-Reply-To: <15528.55377.951848.352486@slothrop.zope.com> References: <15528.55377.951848.352486@slothrop.zope.com> Message-ID: <20020401221143.GA26242@panix.com> On Mon, Apr 01, 2002, Jeremy Hylton wrote: > > I'd appreciate some review to make sure I've not omitted something > that's important. Where is it? Or is it just in CVS? <-- CVS newbie, be careful ;-) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ Why is this newsgroup different from all other newsgroups? From fdrake@acm.org Mon Apr 1 22:23:12 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 1 Apr 2002 17:23:12 -0500 Subject: [Python-Dev] new section on naming and binding in reference manual In-Reply-To: <20020401221143.GA26242@panix.com> References: <15528.55377.951848.352486@slothrop.zope.com> <20020401221143.GA26242@panix.com> Message-ID: <15528.56784.58271.485523@grendel.zope.com> Aahz writes: > Where is it? Or is it just in CVS? <-- CVS newbie, be careful ;-) It was only in CVS when Jeremy sent the note; it's now at: http://www.python.org/dev/doc/devel/ref/ref.html -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From barry@barrys-emacs.org Mon Apr 1 22:36:19 2002 From: barry@barrys-emacs.org (Barry Scott) Date: Mon, 1 Apr 2002 23:36:19 +0100 Subject: [Python-Dev] A Hygienic Macro System in Python? In-Reply-To: Message-ID: <000901c1d9cd$a4907c20$070210ac@LAPDANCE> The dislike for macros is welcome. When can I have a C interface to Python that has no macros? I need to be able to dynamically load python in my application. Python integration is optional. I can look use the GetProcAddress to get a real functions, but I have to know the implementation inside the macros and duplicate that code. If there are a pure procedural interface it would be a lot easier for my type of app. BArry From pobrien@orbtech.com Mon Apr 1 22:56:37 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Mon, 1 Apr 2002 16:56:37 -0600 Subject: [Python-Dev] Evil isinstance() In-Reply-To: <200203312219.g2VMJt124097@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido van Rossum] > > This doesn't seem so much of a problem to me to warrant rejecting > > perfectly good numberlike and stringlike classes' instances, by > > isinstance'ing away. > > Agreed, and I think the solution ought to be sought in asking a > question involving hasattr(). Having recently removed most calls to hasattr() from PyCrust's introspection routines, I'd be curious to get more of your opinion on this issue. Specifically, I removed calls to hasattr() because xmlrpclib always returns true for hasattr(), rendering it effectively meaningless. >>> import xmlrpclib >>> p = xmlrpclib.ServerProxy("http://betty.userland.com", verbose=1) >>> hasattr(p, 'bogus') 1 >>> p.bogus [big error message snipped] I discussed this with Mr. Lundh and ultimately worked around this by removing calls to hasattr(), but I'm still left feeling a bit uncomfortable by this whole situation. Perhaps I was being naive, but I thought I could rely on hasattr(). According to Mr. Lundh: "Having an attribute doesn't mean that you carry out any possible operation on this attribute. In this case, the server proxy will return a method proxy for any possible attribute (hasattr and getattr always succeeds). You won't get an error until you attempt to *call* the method proxy..." This bit of advice seems to fall toward the opposite end of the spectrum from the advice to look to hasattr() for a solution. So I figured I would open this can of worms. I hope this isn't too off-topic. --- Patrick K. O'Brien Orbtech From martin@v.loewis.de Mon Apr 1 23:13:26 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 02 Apr 2002 01:13:26 +0200 Subject: [Python-Dev] Namespaces In-Reply-To: <20020401212232.GA14508@panix.com> References: <002901c1d7f2$8b04f560$2eb53bd0@othello> <20020401025823.GA29354@panix.com> <20020401185657.GA6094@panix.com> <15528.51794.642978.22967@slothrop.zope.com> <20020401212232.GA14508@panix.com> Message-ID: Aahz writes: > I'm fine with "local scope" and "object attributes" to disambiguate > them; I just think it's important that people understand that a name is > a name is a name, and all names live in *some* namespace. That isn't really true: a computed attribute lives in no namespace, instead, some function is invoked to determine the attribute value. Furthermore, some attributes live in multiple namespaces. Given obj.name what namespace is considered to find the name? NOT the namespace of obj, alone - Python also considers the namespace of obj's class (if obj is an instance), of the base classes, etc. OTOH, obj.name = None modifies the namespace of obj (unless name is a computed attribute). Regards, Martin From martin@v.loewis.de Mon Apr 1 23:35:19 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 02 Apr 2002 01:35:19 +0200 Subject: [Python-Dev] A Hygienic Macro System in Python? In-Reply-To: <000901c1d9cd$a4907c20$070210ac@LAPDANCE> References: <000901c1d9cd$a4907c20$070210ac@LAPDANCE> Message-ID: "Barry Scott" writes: > I need to be able to dynamically load python in my > application. Python integration is optional. I can look use the > GetProcAddress to get a real functions, but I have to know the > implementation inside the macros and duplicate that code. If there > are a pure procedural interface it would be a lot easier for my type > of app. Which macros would you like to avoid specifically? Regards, Martin From guido@python.org Tue Apr 2 01:26:28 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 01 Apr 2002 20:26:28 -0500 Subject: [Python-Dev] Evil isinstance() In-Reply-To: Your message of "Mon, 01 Apr 2002 16:56:37 CST." References: Message-ID: <200204020126.g321QSI01619@pcp742651pcs.reston01.va.comcast.net> > [Guido van Rossum] > > > This doesn't seem so much of a problem to me to warrant rejecting > > > perfectly good numberlike and stringlike classes' instances, by > > > isinstance'ing away. > > > > Agreed, and I think the solution ought to be sought in asking a > > question involving hasattr(). > > Having recently removed most calls to hasattr() from PyCrust's > introspection routines, I'd be curious to get more of your opinion > on this issue. Specifically, I removed calls to hasattr() because > xmlrpclib always returns true for hasattr(), rendering it > effectively meaningless. Hm. I didn't know xmlrpc did that, and I think it's pretty unique in this respect. It looks like a hack, and one that can only work as long as attributes on xmlrpc instances are always supposed to be methods. Why didn't you choose to special-case xmlrpc? It appears that introspecting an xmlrpc instance is hopeless anyway... (Depending on what you need the introspection for, of course. You haven't told us the whole story.) It's true, you can't trust hasattr() 100%. In fact, there's no way to determine with 100% accuracy whether an object has a particular attribute or not. But that doesn't mean you can't make an educated guess. --Guido van Rossum (home page: http://www.python.org/~guido/) From brian@sweetapp.com Tue Apr 2 02:01:03 2002 From: brian@sweetapp.com (Brian Quinlan) Date: Mon, 1 Apr 2002 18:01:03 -0800 Subject: [Python-Dev] Evil isinstance() In-Reply-To: <200204020126.g321QSI01619@pcp742651pcs.reston01.va.comcast.net> Message-ID: <003801c1d9ea$3ea2dd00$445d4540@Dell2> Guido wrote: [xmlrpclib creates attributes on demand] > Hm. I didn't know xmlrpc did that, and I think it's pretty unique in > this respect. I've seen other RPC code do this. Some soaplib? objects used to infinitely recurse, if you repred, them due to a subtle __getattr__ bug. > It looks like a hack, and one that can only work as > long as attributes on xmlrpc instances are always supposed to be > methods. I imagine that we are talking about xmlrpclib.Server. This class creates callable objects, representing remote method calls, when attributes are requested. This allows for a pretty natural method call syntax e.g. server = xmlrpclib.Server('http://...') server.do_something_remote() server.do_something_else() XML-RPC does not have an attribute concept. > Why didn't you choose to special-case xmlrpc? It appears > that introspecting an xmlrpc instance is hopeless anyway... > (Depending on what you need the introspection for, of course. You > haven't told us the whole story.) It's not necessarily hopeless but getting a list of available methods (and therefore attribute that should be offered) costs a round-trip and isn't supported by all XML-RPC servers (like Python's SimpleXMLRPCServer, for example). Cheers, Brian From guido@python.org Tue Apr 2 02:14:17 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 01 Apr 2002 21:14:17 -0500 Subject: [Python-Dev] restrictions on global statement In-Reply-To: Your message of "Mon, 01 Apr 2002 16:24:53 EST." <0GTW003O5RHH6H@mtaout01.icomcast.net> References: <0GTW003O5RHH6H@mtaout01.icomcast.net> Message-ID: <200204020214.g322ENZ02333@pcp742651pcs.reston01.va.comcast.net> > The lang ref says: > > Names listed in a \keyword{global} statement must not be defined > as formal parameters or in a \keyword{for} loop control target, > \keyword{class} definition, function definition, or > \keyword{import} statement. > > (The current implementation does not enforce the latter two > restrictions, but programs should not abuse this freedom, as > future implementations may enforce them or silently change the > meaning of the program.) > > While this is all technically correct, the second paragraph is a bit > misleading. The only case enforced by the compiler is that a formal > parameter can't be declared global. > > Do we actually care about this rule? The lang ref would be shorter if > it were silent on the issue, and it would more closely describe the > way Python has behaved as long as I've been using it. > > Jeremy I believe at the time when this was added, we had a good reason for it. But I don't recall what it was. It may have had to do with some tools that do an imperfect parse (e.g. looking for 'class' but not scanning for 'global'). I'm not sure it's worth keeping. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 2 02:41:45 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 01 Apr 2002 21:41:45 -0500 Subject: [Python-Dev] Re: PEP 279 In-Reply-To: Your message of "Sun, 31 Mar 2002 21:20:48 PST." <3CA7EE30.55A52@ActiveState.com> References: <3CA7EE30.55A52@ActiveState.com> Message-ID: <200204020241.g322fjj03633@pcp742651pcs.reston01.va.comcast.net> > [RDH] > > > Executive Summary: > > > 1. iterindexed(collection) --> accepted > [GvR] > > Except I want to think more about the name. > > Okay, here's what we have so far: > > iterindexed()-- five syllables is a mouthfull > index() -- nice verb but could be confused the .index() method > indexed() -- widely liked however adjectives should be avoided > count() -- direct and explicit but often used in other contexts > itercount() -- direct, explicit and hated by more than one person > enumerate() -- a contender but doesn't mention iteration or indices > iteritems() -- already used by dictionaries for key:value pairs [David Ascher] > My current fave is 'indexer'. Something which creates an index. I like it! Raymond, what do you think? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 2 04:05:52 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 01 Apr 2002 23:05:52 -0500 Subject: [Python-Dev] Re: PEP 279 In-Reply-To: Your message of "Mon, 01 Apr 2002 21:41:45 EST." Message-ID: <200204020405.g3245wv04622@pcp742651pcs.reston01.va.comcast.net> After some more thinking about the name, I have two contenders left: enumerate() and indexer(). Let me explain why I reject the others: > iterindexed()-- five syllables is a mouthfull Indeed. > index() -- nice verb but could be confused the .index() method Indeed. > indexed() -- widely liked however adjectives should be avoided Indeed. > count() -- direct and explicit but often used in other contexts In particular, there's a list method by this name. While that's in a different namespace, I think the core language should be careful not to pile too many meanings on the same name. > itercount() -- direct, explicit and hated by more than one person Did they explain why they hated it? "Hate it" alone doesn't get much credit in my book. > iteritems() -- already used by dictionaries for key:value pairs Which is a downside to me. The symmetry between (key:value) for mappings and (index:value) for sequences seems appealing but quickly becomes a problem, e.g. "for i in " iterates over the values but "for i in " iterates over the keys. So now I'd like to choose between enumerate() and indexer(). Any closing arguments? --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Tue Apr 2 04:22:01 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 1 Apr 2002 23:22:01 -0500 Subject: [Python-Dev] Re: PEP 279 In-Reply-To: <200204020405.g3245wv04622@pcp742651pcs.reston01.va.comcast.net> References: <200204020405.g3245wv04622@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15529.12777.4579.553149@grendel.zope.com> Guido van Rossum writes: > So now I'd like to choose between enumerate() and indexer(). Any > closing arguments? "indexer" is the name of the built-in full-text indexer, right? ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From barry@zope.com Tue Apr 2 05:11:56 2002 From: barry@zope.com (Barry A. Warsaw) Date: Tue, 2 Apr 2002 00:11:56 -0500 Subject: [Python-Dev] Re: PEP 279 References: <200204020405.g3245wv04622@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15529.15772.187073.360462@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> So now I'd like to choose between enumerate() and indexer(). GvR> Any closing arguments? Let me enumerate the reasons while I like the one I do: 1. it clearly describes the action being performed 2. see #1 3. it is a verb, which (to me) seems more preferable than a noun 4. there is /no/ ... number 4 5. indexer() seems too tied to search engines 6. Need I say more? :) -Barry From tim.one@comcast.net Tue Apr 2 05:14:22 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 02 Apr 2002 00:14:22 -0500 Subject: [Python-Dev] Re: PEP 279 In-Reply-To: <200204020405.g3245wv04622@pcp742651pcs.reston01.va.comcast.net> Message-ID: > So now I'd like to choose between enumerate() and indexer(). Any > closing arguments? I'm leaving that quote anonymous so nobody gives it more or less weight than it deserves . I prefer enumerate(), because a transitive verb is more appropriate than a noun. OTOH, enumerate() is just a fancy pants way of saying "countoff()", which is nicely confusable with operator.countOf if you're tired . From DavidA@ActiveState.com Tue Apr 2 05:24:54 2002 From: DavidA@ActiveState.com (David Ascher) Date: Mon, 01 Apr 2002 21:24:54 -0800 Subject: [Python-Dev] Re: PEP 279 References: <200204020405.g3245wv04622@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CA940A6.B91D0FBF@ActiveState.com> Guido van Rossum wrote: > So now I'd like to choose between enumerate() and indexer(). Any > closing arguments? A few thoughts: + the nice thing about enumerate is that it has the word 'number' in it. - it is not obvious to me at first blush that 'enumerating' and 'iterating' elicit different 'first-guess semantics' - The first meaning I'd guess for enumerate(foo) would be range(len(foo)). One reason I suggested 'indexer' is that the tuples (index, element) are what I think of as an index entry -- it's a pair of (key, the element being located) A problem with indexer that I just realized is that indexer(dictionary) does return (number, value) pairs, while one might reasonably expect it to generate the same thing as dictionary.iteritems(). >>> print d {'y': 'foo', 'x': 123, 'z': 1j} >>> for (k,v) in indexer(d): ... print k, v ... 0 y 1 x 2 z >>> for (k,v) in d.iteritems(): ... print k, v ... y foo x 123 z 1j >>> for (k,v) in enumerate(d): ... print k, v ... 0 y 1 x 2 z Actually, having the function return the dictionary _keys_ as the second element in the tuple is a little weird. But I guess that's just the consequence of the earlier choice w.r.t. iter(dict). All things considered, even though I suggested 'indexer()', I'm voting for 'enumerate()' because of the numeric focus as opposed to a 'key' focus. --da From goodger@users.sourceforge.net Tue Apr 2 05:28:17 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Tue, 02 Apr 2002 00:28:17 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format Message-ID: Here's a serious proposal, safe to post now that April Fool's is over. Please read & comment. -- David Goodger goodger@users.sourceforge.net Open-source projects: - Python Docstring Processing System: http://docstring.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net - The Go Tools Project: http://gotools.sourceforge.net PEP: 287 Title: reStructuredText Standard Docstring Format Version: $Revision: 1.3 $ Last-Modified: $Date: 2002/04/02 03:50:38 $ Author: goodger@users.sourceforge.net (David Goodger) Discussions-To: doc-sig@python.org Status: Draft Type: Informational Created: 25-Mar-2002 Post-History: 02-Apr-2002 Replaces: 216 Abstract When plaintext hasn't been expressive enough for inline documentation, Python programmers have sought out a format for docstrings. This PEP proposes that the reStructuredText markup [1]_ be adopted as the standard markup format for structured plaintext documentation in Python docstrings, and for PEPs and ancillary documents as well. reStructuredText is a rich and extensible yet easy-to-read, what-you-see-is-what-you-get plaintext markup syntax. Only the low-level syntax of docstrings is addressed here. This PEP is not concerned with docstring semantics or processing at all. Nor is it an attempt to deprecate pure plaintext docstrings, which are always going to be legitimate. The reStructuredText markup is an alternative for those who want more expressive docstrings. Benefits Programmers are by nature a lazy breed. We reuse code with functions, classes, modules, and subsystems. Through its docstring syntax, Python allows us to document our code from within. The "holy grail" of the Python Documentation Special Interest Group (Doc-SIG) [2]_ has been a markup syntax and toolset to allow auto-documentation, where the docstrings of Python systems can be extracted in context and processed into useful, high-quality documentation for multiple purposes. The proposed format (reStructuredText) is entirely readable in plaintext format, and many of the markup forms match common usage (e.g., ``*emphasis*``), so it reads quite naturally. Yet it is rich enough to produce complex documents, and extensible so that there are few limits. The reStructuredText parser is available now. The Docutils project is at the point where standalone reStructuredText documents can be converted to HTML; other output format writers will become available over time. Work is progressing on a Python source "Reader" which will implement auto-documentation. Authors of existing auto-documentation tools are encouraged to integrate the reStructuredText parser into their projects, or better yet, to join forces to produce a world-class toolset for the Python standard library. Tools will become available in the near future, which will allow programmers to generate HTML for online help, XML for multiple purposes, and perhaps eventually PDF/DocBook/LaTeX for printed documentation, essentially "for free" from the existing docstrings. The adoption of a standard will, at the very least, benefit docstring processing tools by preventing further "reinventing the wheel". Eventually PyDoc, the one existing standard auto-documentation tool, could have reStructuredText support added. In the interim it will have no problem with reStructuredText markup, since it treats all docstrings as plaintext. Goals These are the generally accepted goals for a docstring format, as discussed in the Doc-SIG: 1. It must be readable in source form by the casual observer. 2. It must be easy to type with any standard text editor. 3. It must not need to contain information which can be deduced from parsing the module. 4. It must contain sufficient information (structure) so it can be converted to any reasonable markup format. 5. It must be possible to write a module's entire documentation in docstrings, without feeling hampered by the markup language. reStructuredText meets and exceeds all of these goals, and sets its own goals as well, even more stringent. See "Features" below. The goals of this PEP are as follows: 1. To establish reStructuredText as a standard docstring format by attaining "accepted" status (Python community consensus; BDFL pronouncement). Once reStructuredText is a Python standard, effort can be focused on tools instead of arguing for a standard. Python needs a standard set of documentation tools. 2. To address any related concerns raised by the Python community. 3. To encourage community support. As long as multiple competing markups are out there, the development community remains fractured. Once a standard exists, people will start to use it, and momentum will inevitably gather. 4. To consolidate efforts from related auto-documentation projects. It is hoped that interested developers will join forces and work on a joint/merged/common implementation. 5. To adopt reStructuredText as the standard markup for PEPs. One or both of the following strategies may be applied: a) Keep the existing PEP section structure constructs (one-line section headers, indented body text). Subsections can either be forbidden or supported with underlined headers in the indented body text. b) Replace the PEP section structure constructs with the reStructuredText syntax. Section headers will require underlines, subsections will be supported out of the box, and body text need not be indented (except for block quotes). Support for RFC 2822 headers will be added to the reStructuredText parser (unambiguous given a specific context: the first contiguous block of a PEP document). It may be desired to concretely specify what over/underline styles are allowed for PEP section headers, for uniformity. 6. To adopt reStructuredText as the standard markup for README-type files and other standalone documents in the Python distribution. Rationale The lack of a standard syntax for docstrings has hampered the development of standard tools for extracting and converting docstrings into documentation in standard formats (e.g., HTML, DocBook, TeX). There have been a number of proposed markup formats and variations, and many tools tied to these proposals, but without a standard docstring format they have failed to gain a strong following and/or floundered half-finished. Throughout the existence of the Doc-SIG, consensus on a single standard docstring format has never been reached. A lightweight, implicit markup has been sought, for the following reasons (among others): 1. Docstrings written within Python code are available from within the interactive interpreter, and can be 'print'ed. Thus the use of plaintext for easy readability. 2. Programmers want to add structure to their docstrings, without sacrificing raw docstring readability. Unadorned plaintext cannot be transformed ('up-translated') into useful structured formats. 3. Explicit markup (like XML or TeX) is widely considered unreadable by the uninitiated. 4. Implicit markup is aesthetically compatible with the clean and minimalist Python syntax. Proposed alternatives have included: - XML [3]_, SGML [4]_, DocBook [5]_, HTML [6]_, XHTML [7]_ XML and SGML are explicit, well-formed meta-languages suitable for all kinds of documentation. XML is a variant of SGML. They are best used behind the scenes, because they are verbose, difficult to type, and too cluttered to read comfortably as source. DocBook, HTML, and XHTML are all applications of SGML and/or XML, and all share the same basic syntax and the same shortcomings. - TeX [8]_ TeX is similar to XML/SGML in that it's explicit, not very easy to write, and not easy for the uninitiated to read. - Perl POD [9]_ Most Perl modules are documented in a format called POD -- Plain Old Documentation. This is an easy-to-type, very low level format with strong integration with the Perl parser. Many tools exist to turn POD documentation into other formats: info, HTML and man pages, among others. However, the POD syntax takes after Perl itself in terms of readability. - JavaDoc [10]_ Special comments before Java classes and functions serve to document the code. A program to extract these, and turn them into HTML documentation is called javadoc, and is part of the standard Java distribution. However, the only output format that is supported is HTML, and JavaDoc has a very intimate relationship with HTML, using HTML tags for most markup. Thus it shares the readability problems of HTML. - Setext [11]_, StructuredText [12]_ Early on, variants of Setext (Structure Enhanced Text), including Zope Corp's StructuredText, were proposed for Python docstring formatting. Hereafter these variants will collectively be call 'STexts'. STexts have the advantage of being easy to read without special knowledge, and relatively easy to write. Although used by some (including in most existing Python auto-documentation tools), until now STexts have failed to become standard because: - STexts have been incomplete. Lacking "essential" constructs that people want to use in their docstrings, STexts are rendered less than ideal. Note that these "essential" constructs are not universal; everyone has their own requirements. - STexts have been sometimes surprising. Bits of text are unexpectedly interpreted as being marked up, leading to user frustration. - SText implementations have been buggy. - Most STexts have have had no formal specification except for the implementation itself. A buggy implementation meant a buggy spec, and vice-versa. - There has been no mechanism to get around the SText markup rules when a markup character is used in a non-markup context. Proponents of implicit STexts have vigorously opposed proposals for explicit markup (XML, HTML, TeX, POD, etc.), and the debates have continued off and on since 1996 or earlier. reStructuredText is a complete revision and reinterpretation of the SText idea, addressing all of the problems listed above. Features Rather than repeating or summarizing the extensive reStructuredText spec, please read the originals available from http://structuredtext.sourceforge.net/spec/ (.txt & .html files). Reading the documents in following order is recommended: - An Introduction to reStructuredText [13]_ - Problems With StructuredText [14]_ (optional for those who have used StructuredText; it explains many markup decisions made) - reStructuredText Markup Specification [15]_ - A Record of reStructuredText Syntax Alternatives [16]_ (explains markup decisions made independently of StructuredText) - reStructuredText Directives [17]_ There is also a "Quick reStructuredText" user reference [18]_. A summary of features addressing often-raised docstring markup concerns follows: - A markup escaping mechanism. Backslashes (``\``) are used to escape markup characters when needed for non-markup purposes. However, the inline markup recognition rules have been constructed in order to minimize the need for backslash-escapes. For example, although asterisks are used for *emphasis*, in non-markup contexts such as "*" or "(*)" or "x * y", the asterisks are not interpreted as markup and are left unchanged. For many non-markup uses of backslashes (e.g., describing regular expressions), inline literals or literal blocks are applicable; see the next item. - Markup to include Python source code and Python interactive sessions: inline literals, literal blocks, and doctest blocks. Inline literals use ``double-backquotes`` to indicate program I/O or code snippets. No markup interpretation (including backslash-escape [``\``] interpretation) is done within inline literals. Literal blocks (block-level literal text, such as code excerpts or ASCII graphics) are indented, and indicated with a double-colon ("::") at the end of the preceding paragraph (right here -->):: if literal_block: text = 'is left as-is' spaces_and_linebreaks = 'are preserved' markup_processing = None Doctest blocks begin with ">>> " and end with a blank line. Neither indentation nor literal block double-colons are required. For example:: Here's a doctest block: >>> print 'Python-specific usage examples; begun with ">>>"' Python-specific usage examples; begun with ">>>" >>> print '(cut and pasted from interactive sessions)' (cut and pasted from interactive sessions) - Markup that isolates a Python identifier: interpreted text. Text enclosed in single backquotes is recognized as "interpreted text", whose interpretation is application-dependent. In the context of a Python docstring, the default interpretation of interpreted text is as Python identifiers. The text will be marked up with a hyperlink connected to the documentation for the identifier given. Lookup rules are the same as in Python itself: LGB namespace lookups (local, global, builtin). The "role" of the interpreted text (identifying a class, module, function, etc.) is determined implicitly from the namespace lookup. For example:: class Keeper(Storer): """ Keep data fresher longer. Extend `Storer`. Class attribute `instances` keeps track of the number of `Keeper` objects instantiated. """ instances = 0 """How many `Keeper` objects are there?""" def __init__(self): """ Extend `Storer.__init__()` to keep track of instances. Keep count in `self.instances` and data in `self.data`. """ Storer.__init__(self) self.instances += 1 self.data = [] """Store data in a list, most recent last.""" def storedata(self, data): """ Extend `Storer.storedata()`; append new `data` to a list (in `self.data`). """ self.data = data Each piece of interpreted text is looked up according to the local namespace of the block containing its docstring. - Markup that isolates a Python identifier and specifies its type: interpreted text with roles. Although the Python source context reader is designed not to require explicit roles, they may be used. To classify identifiers explicitly, the role is given along with the identifier in either prefix or suffix form:: Use :method:`Keeper.storedata` to store the object's data in `Keeper.data`:instance_attribute:. The syntax chosen for roles is verbose, but necessarily so (if anyone has a better alternative, please post it to the Doc-SIG). The intention of the markup is that there should be little need to use explicit roles; their use is to be kept to an absolute minimum. - Markup for "tagged lists" or "label lists": field lists. Field lists represent a mapping from field name to field body. These are mostly used for extension syntax, such as "bibliographic field lists" (representing document metadata such as author, date, and version) and extension attributes for directives (see below). They may be used to implement docstring semantics, such as identifying parameters, exceptions raised, etc.; such usage is beyond the scope of this PEP. A modified RFC 2822 syntax is used, with a colon *before* as well as *after* the field name. Field bodies are more versatile as well; they may contain multiple field bodies (even nested field lists). For example:: :Date: 2002-03-22 :Version: 1 :Authors: - Me - Myself - I Standard RFC 2822 header syntax cannot be used for this construct because it is ambiguous. A word followed by a colon at the beginning of a line is common in written text. - Markup extensibility: directives and substitutions. Directives are used as an extension mechanism for reStructuredText, a way of adding support for new block-level constructs without adding new syntax. Directives for images, admonitions (note, caution, etc.), and tables of contents generation (among others) have been implemented. For example, here's how to place an image:: .. image:: mylogo.png Substitution definitions allow the power and flexibility of block-level directives to be shared by inline text. For example:: The |biohazard| symbol must be used on containers used to dispose of medical waste. .. |biohazard| image:: biohazard.png - Section structure markup. Section headers in reStructuredText use adornment via underlines (and possibly overlines) rather than indentation. For example:: This is a Section Title ======================= This is a Subsection Title -------------------------- This paragraph is in the subsection. This is Another Section Title ============================= This paragraph is in the second section. Questions & Answers Q1: Is reStructuredText rich enough? A1: Yes, it is for most people. If it lacks some construct that is required for a specific application, it can be added via the directive mechanism. If a common construct has been overlooked and a suitably readable syntax can be found, it can be added to the specification and parser. Q2: Is reStructuredText *too* rich? A2: For specific applications or individuals, perhaps. In general, no. Since the very beginning, whenever a markup syntax has been proposed on the Doc-SIG, someone has complained about the lack of support for some construct or other. The reply was often something like, "These are docstrings we're talking about, and docstrings shouldn't have complex markup." The problem is that a construct that seems superfluous to one person may be absolutely essential to another. reStructuredText takes the opposite approach: it provides a rich set of implicit markup constructs (plus a generic extension mechanism for explicit markup), allowing for all kinds of documents. If the set of constructs is too rich for a particular application, the unused constructs can either be removed from the parser (via application-specific overrides) or simply omitted by convention. Q3: Why not use indentation for section structure, like StructuredText does? Isn't it more "Pythonic"? A3: Guido van Rossum wrote the following in a 2001-06-13 Doc-SIG post: I still think that using indentation to indicate sectioning is wrong. If you look at how real books and other print publications are laid out, you'll notice that indentation is used frequently, but mostly at the intra-section level. Indentation can be used to offset lists, tables, quotations, examples, and the like. (The argument that docstrings are different because they are input for a text formatter is wrong: the whole point is that they are also readable without processing.) I reject the argument that using indentation is Pythonic: text is not code, and different traditions and conventions hold. People have been presenting text for readability for over 30 centuries. Let's not innovate needlessly. See "Section Structure via Indentation" in "Problems With StructuredText" [14]_ for further elaboration. Q4: Why use reStructuredText for PEPs? What's wrong with the existing standard? A4: The existing standard for PEPs is very limited in terms of general expressibility, and referencing is especially lacking for such a reference-rich document type. PEPs are currently converted into HTML, but the results (mostly monospaced text) are less than attractive, and most of the value-added potential of HTML is untapped. Making reStructuredText the standard markup for PEPs will enable much richer expression, including support for section structure, inline markup, graphics, and tables. In several PEPs there are ASCII graphics diagrams, which are all that plaintext documents can support. Since PEPs are made available in HTML form, the ability to include proper diagrams would be immediately useful. Current PEP practices allow for reference markers in the form "[1]" in the text, and the footnotes/references themselves are listed in a section toward the end of the document. There is currently no hyperlinking between the reference marker and the footnote/reference itself (it would be possible to add this to pep2html.py, but the "markup" as it stands is ambiguous and mistakes would be inevitable). A PEP with many references (such as this one ;-) requires a lot of flipping back and forth. When revising a PEP, often new references are added or unused references deleted. It is painful to renumber the references, since it has to be done in two places and can have a cascading effect (insert a single new reference 1, and every other reference has to be renumbered; always adding new references to the end is suboptimal). It is easy for references to go out of sync. PEPs use references for two purposes: simple URL references and footnotes. reStructuredText differentiates between the two. A PEP might contain references like this:: Abstract This PEP proposes adding frungible doodads [1] to the core. It extends PEP 9876 [2] via the BCA [3] mechanism. References and Footnotes [1] http://www.example.org/ [2] PEP 9876, Let's Hope We Never Get Here http://www.python.org/peps/pep-9876.html [3] "Bogus Complexity Addition" Reference 1 is a simple URL reference. Reference 2 is a footnote containing text and a URL. Reference 3 is a footnote containing text only. Rewritten using reStructuredText, this PEP could look like this:: Abstract ======== This PEP proposes adding `frungible doodads`_ to the core. It extends PEP 9876 [#pep9876]_ via the BCA [#]_ mechanism. .. _frungible doodads: http://www.example.org/ .. [#pep9876] `PEP 9876`__, Let's Hope We Never Get Here __ http://www.python.org/peps/pep-9876.html .. [#] "Bogus Complexity Addition" URLs and footnotes can be defined close to their references if desired, making them easier to read in the source text, and making the PEPs easier to revise. The "References and Footnotes" section can be auto-generated with a document tree transform. Footnotes from throughout the PEP would be gathered and displayed under a standard header. If URL references should likewise be written out explicitly (in citation form), another tree transform could be used. URL references can be named ("frungible doodads"), and can be referenced from multiple places in the document without additional definitions. When converted to HTML, references will be replaced with inline hyperlinks (HTML tags). The two footnotes are automatically numbered, so they will always stay in sync. The first footnote also contains an internal reference name, "pep9876", so it's easier to see the connection between reference and footnote in the source text. Named footnotes can be referenced multiple times, maintaining consistent numbering. The "#pep9876" footnote could also be written in the form of a citation:: It extends PEP 9876 [PEP9876]_ ... .. [PEP9876] `PEP 9876`_, Let's Hope We Never Get Here Footnotes are numbered, whereas citations use text for their references. Q5: Wouldn't it be better to keep the docstring and PEP proposals separate? A5: The PEP markup proposal may be removed if it is deemed that there is no need for PEP markup, or it could be made into a separate PEP. If accepted, PEP 1, PEP Purpose and Guidelines [19]_, and PEP 9, Sample PEP Template [20]_ will be updated. It seems natural to adopt a single consistent markup standard for all uses of structured plaintext in Python, and to propose it all in one place. Q6: The existing pep2html.py script converts the existing PEP format to HTML. How will the new-format PEPs be converted to HTML? A6: One of the deliverables of the Docutils project [21]_ will be a new version of pep2html.py with integrated reStructuredText parsing. The Docutils project will support PEPs with a "PEP Reader" component, including all functionality currently in pep2html.py (auto-recognition of PEP & RFC references). Q7: Who's going to convert the existing PEPs to reStructuredText? A7: A call for volunteers will be put out to the Doc-SIG and greater Python communities. If insufficient volunteers are forthcoming, I (David Goodger) will convert the documents myself, perhaps with some level of automation. A transitional system whereby both old and new standards can coexist will be easy to implement (and I pledge to implement it if necessary). Q8: Why use reStructuredText for README and other ancillary files? A8: The reasoning given for PEPs in A4 above also applies to README and other ancillary files. By adopting a standard markup, these files can be converted to attractive cross-referenced HTML and put up on python.org. Developers of Python projects can also take advantage of this facility for their own documentation. Q9: Won't the superficial similarity to existing markup conventions cause problems, and result in people writing invalid markup (and not noticing, because the plaintext looks natural)? How forgiving is reStructuredText of "not quite right" markup? A9: There will be some mis-steps, as there would be when moving from one programming language to another. As with any language, proficiency grows with experience. Luckily, reStructuredText is a very little language indeed. As with any syntax, there is the possibility of syntax errors. It is expected that a user will run the processing system over their input and check the output for correctness. In a strict sense, the reStructuredText parser is very unforgiving (as it should be; "In the face of ambiguity, refuse the temptation to guess" [22]_ applies to parsing markup as well as computer languages). Here's a design goal from "An Introduction to reStructuredText" [13]_: 3. Unambiguous. The rules for markup must not be open for interpretation. For any given input, there should be one and only one possible output (including error output). While unforgiving, at the same time the parser does try to be helpful by producing useful diagnostic output ("system messages"). The parser reports problems, indicating their level of severity (from least to most: debug, info, warning, error, severe). The user or the client software can decide on reporting thresholds; they can ignore low-level problems or cause high-level problems to bring processing to an immediate halt. Problems are reported during the parse as well as included in the output, often with two-way links between the source of the problem and the system message explaining it. Q10: Will the docstrings in the Python standard library modules be converted to reStructuredText? A10: Over time, with the help of the developer community, many modules will be converted. Some modules may never be converted. A future toolset will have to allow for incompleteness. References & Footnotes [1] http://structuredtext.sourceforge.net/ [2] http://www.python.org/sigs/doc-sig/ [3] http://www.w3.org/XML/ [4] http://www.oasis-open.org/cover/general.html [5] http://docbook.org/tdg/en/html/docbook.html [6] http://www.w3.org/MarkUp/ [7] http://www.w3.org/MarkUp/#xhtml1 [8] http://www.tug.org/interest.html [9] http://www.perldoc.com/perl5.6/pod/perlpod.html [10] http://java.sun.com/j2se/javadoc/ [11] http://docutils.sourceforge.net/mirror/setext.html [12] http://dev.zope.org/Members/jim/StructuredTextWiki/FrontPage [13] An Introduction to reStructuredText http://structuredtext.sourceforge.net/spec/introduction.txt [14] Problems with StructuredText http://structuredtext.sourceforge.net/spec/problems.txt [15] reStructuredText Markup Specification http://structuredtext.sourceforge.net/spec/reStructuredText.txt [16] A Record of reStructuredText Syntax Alternatives http://structuredtext.sourceforge.net/spec/alternatives.txt [17] reStructuredText Directives http://structuredtext.sourceforge.net/spec/directives.txt [18] Quick reStructuredText http://structuredtext.sourceforge.net/docs/quickref.html [19] PEP 1, PEP Guidelines, Warsaw, Hylton http://www.python.org/peps/pep-0001.html [20] PEP 9, Sample PEP Template, Warsaw http://www.python.org/peps/pep-0009.html [21] http://docutils.sourceforge.net/ [22] From "The Zen of Python (by Tim Peters)", http://www.python.org/doc/Humor.html#zen [23] PEP 216, Docstring Format, Zadka http://www.python.org/peps/pep-0216.html Copyright This document has been placed in the public domain. Acknowledgements Some text is borrowed from PEP 216, Docstring Format [23]_, by Moshe Zadka. Special thanks to all members past & present of the Python Doc-SIG. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: From python@rcn.com Tue Apr 2 05:28:45 2002 From: python@rcn.com (Raymond Hettinger) Date: Tue, 2 Apr 2002 00:28:45 -0500 Subject: [Python-Dev] Re: PEP 279 References: Message-ID: <002d01c1da07$434c88c0$4c66accf@othello> From: "Tim Peters" > > So now I'd like to choose between enumerate() and indexer(). Any > > closing arguments? > > I'm leaving that quote anonymous so nobody gives it more or less weight than > it deserves . I prefer enumerate(), because a transitive verb is more > appropriate than a noun. OTOH, enumerate() is just a fancy pants way of > saying "countoff()", which is nicely confusable with operator.countOf if > you're tired . > I agree. Enumerate is a direct, unequivocable verb that speaks directly to the idea of sequentially assigning numbers to things. Countoff is cool, but is a little colloquial. Also, any of the words incorporating count (including itercount and countoff) have a subtle implicit suggestion of counting from one rather than zero -- "firstly, let's take item zero ..." While indices have a clear meaning in Python, the old Dbase and SQL part of me is a little bugged by index(), indexed(), indexer() since sequential files are not at all like indexed files. When I index a database, sorting takes place. When I index a collection, sequencing is applied. Much different. As a part of speech, indexer() doesn't work as well in a sentence: for linenum, line in indexer(file): ... # yuck! Summary, I think we're down to subtle differences and have the following preference order: 1. enumerate 2. countoff 3. indexed 4. itercount OTOH, no one has ever accused me of having good taste Raymond Hettinger From python@rcn.com Tue Apr 2 05:32:08 2002 From: python@rcn.com (Raymond Hettinger) Date: Tue, 2 Apr 2002 00:32:08 -0500 Subject: [Python-Dev] Re: PEP 279 References: <200204020405.g3245wv04622@pcp742651pcs.reston01.va.comcast.net> <3CA940A6.B91D0FBF@ActiveState.com> Message-ID: <003301c1da07$bc1b0c40$4c66accf@othello> From: "David Ascher" > + the nice thing about enumerate is that it has the word 'number' in > All things considered, even though I suggested 'indexer()', I'm voting > for 'enumerate()' because of the numeric focus as opposed to a 'key' > focus. Yes! -- Raymond From tim.one@comcast.net Tue Apr 2 05:46:46 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 02 Apr 2002 00:46:46 -0500 Subject: [Python-Dev] Re: PEP 279 In-Reply-To: <002d01c1da07$434c88c0$4c66accf@othello> Message-ID: [Raymond Hettinger] > ... > Summary, I think we're down to subtle differences and have the following > preference order: > 1. enumerate > 2. countoff > 3. indexed > 4. itercount No, no, no. That's not the Python Way. These aren't subtle differences, "enumerate" is the correct choice and all the others are plain *stupid*. TOOWTDI! Once you embrace the blinding obviousness of that, you'll have no trouble at all understanding why extending generators meets with so much otherwise inexplicable resistance . > OTOH, no one has ever accused me of having good taste You're in danger of them starting to, provided Guido picks enumerate. Then people will remember that you channeled him successfully. But if he doesn't pick enumerate, all they'll remember is that you ran around half insane, accusing everyone who didn't vote for the loser "enumerate" of trying to destroy Python. This is a very delicate time in your Python Career. doubting-anyone-will-get-much-sleep-tonight-ly y'rs - tim From python@rcn.com Tue Apr 2 06:20:39 2002 From: python@rcn.com (Raymond Hettinger) Date: Tue, 2 Apr 2002 01:20:39 -0500 Subject: [Python-Dev] Re: PEP 279 References: Message-ID: <000f01c1da0e$82fe8480$4c66accf@othello> From: "Tim Peters" > > No, no, no. That's not the Python Way. These aren't subtle differences, > "enumerate" is the correct choice and all the others are plain *stupid*. > TOOWTDI! Once you embrace the blinding obviousness of that, you'll have no > trouble at all understanding why extending generators meets with so much > otherwise inexplicable resistance . Of course, how could I have been so blind. > > > OTOH, no one has ever accused me of having good taste > > You're in danger of them starting to, provided Guido picks enumerate. Then > people will remember that you channeled him successfully. But if he doesn't > pick enumerate, all they'll remember is that you ran around half insane, > accusing everyone who didn't vote for the loser "enumerate" of trying to > destroy Python. This is a very delicate time in your Python Career. The scales are starting to tip. But which way? > doubting-anyone-will-get-much-sleep-tonight-ly y'rs - tim > Now, that's a great idea. See you guys tomorrow. It's time for me to enumerate(sheep). Raymond From mal@lemburg.com Tue Apr 2 09:18:55 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 02 Apr 2002 11:18:55 +0200 Subject: [Python-Dev] Deprecating C APIs (Use of PyArg_NoArgs()) References: <3C9F4AC0.C076679B@metaslash.com> <3C9F8B0F.D921E530@metaslash.com> <200203252052.g2PKqJa08509@pcp742651pcs.reston01.va.comcast.net> <3C9FA9D1.DA46BE04@metaslash.com> <200203252300.g2PN0DY09544@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CA9777F.BB3DA0DD@lemburg.com> Guido van Rossum wrote: > > > Both PyArg_GetInt() & PyArg_NoArgs() both have comments above them > > stating they should not be used. There are no more occurances > > of PyArg_GetInt(). I'm working on reducing METH_OLDARGS, down to 214. > > > > If you want I can remove them from Include/Python.h when done. > > Has the api documentation documented them as obsolete? If so, or if > they were undocumented, yes, please remove them. If not, I'd see if > there's a way to turn these into warnings for one release. +1 on removing PyArg_GetInt() -1 on removing PyArg_NoArgs() ... this API is still in very active use in third party extensions. > > If so, should I write up anything in NEWS? > > Yes, I think so. It's always good to announce explicitly when > a previously deprecated item actually gets deleted, no matter how > small. I think we should come up with a better way to deprecate C level APIs. NEWS is not as effective as one might wish; any deprecation or removal of an API should also include upgrade information for the C extension writer so that upgrading their modules does not require diving into the new style C API code every time. I'd suggest to have a PEP which documents all C API changes or deprecations together with a short decsription of how to update old code. Another aspect which we ought to consider is that C extension writers may have a different view of when to leave backward compatibility path in favour of a new Python version. IMHO, there should always be a way to emulate old behaviour so that extensions can continue to support Python 1.5.2 if they need to. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From ping@lfw.org Tue Apr 2 11:25:45 2002 From: ping@lfw.org (Ka-Ping Yee) Date: Tue, 2 Apr 2002 05:25:45 -0600 (CST) Subject: [Python-Dev] Re: PEP 279 In-Reply-To: <000f01c1da0e$82fe8480$4c66accf@othello> Message-ID: On Tue, 2 Apr 2002, Raymond Hettinger wrote: > Now, that's a great idea. See you guys tomorrow. It's time for me to > enumerate(sheep). I prefer "enumerate" to "indexer" as well, though it's a bit long. "enum" would be nice. (I don't really see how it would ever collide with C's meaning for "enum" -- the current idiom for simulating enumerated types is very different, and even if Python eventually gets a type system fancy enough to handle types with alternatives, we won't use "enum" to describe them -- we'll just have alternatives, and probably use "|" or something like that.) -- ?!ng From guido@python.org Tue Apr 2 13:36:45 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 02 Apr 2002 08:36:45 -0500 Subject: [Python-Dev] Re: PEP 279 In-Reply-To: Your message of "Mon, 01 Apr 2002 23:14:29 EST." <004b01c1d9fd$8be15c00$0202a8c0@boostconsulting.com> References: <200204020405.g3245wv04622@pcp742651pcs.reston01.va.comcast.net> <004b01c1d9fd$8be15c00$0202a8c0@boostconsulting.com> Message-ID: <200204021336.g32Dakb05964@pcp742651pcs.reston01.va.comcast.net> [David Abrahams] > I missed what this function does, x(seq) returns [(0, seq[0]), (1, seq[1]), ...] > but: > > "indexer" is a noun. I like it provided the function returns an > indexer(). Otherwise, I'm not too sure. Yeah, that's my main problem with it. > "enumerate()" sounds like it returns a list of the elements, or is > an iterator interface over the elements. Looking it up at > http://www.m-w.com/cgi-bin/dictionary > shows that it is dangerously close to meaning what we want to say. I say > "dangerously" because it *doesn't* mean what we want to say, and I think > a close-but-not-quite meaning is probably much worse than calling it > wafflebird. I don't know. > unfortunately, numerate and itemize both mean about the same... Numerate is obscure, but itemize() has potential! --Guido van Rossum (home page: http://www.python.org/~guido/) From lac@strakt.com Tue Apr 2 13:40:46 2002 From: lac@strakt.com (Laura Creighton) Date: Tue, 02 Apr 2002 15:40:46 +0200 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: Your message of "Sat, 30 Mar 2002 08:35:33 GMT." <2zRcLUAVjXp8EwVv@jessikat.fsnet.co.uk> References: <2zRcLUAVjXp8EwVv@jessikat.fsnet.co.uk> Message-ID: <200204021340.g32Dekrq025410@ratthing-b246.strakt.com> Dear Guido: I would first like to thank you for giving us an opportunity to respond. I have spent most of the weekend thinking and writing a reply to this, and I think that this has made me a better teacher. For this I am grateful. I realise that this is rather long, but I have condensed it substantially from my efforts earlier this weekend. Thank you for taking the time to read this. Laura Creighton ----------------------------- I am opposed to the addition of the new proposed type on the grounds that it will make Python harder to teach both to people who have never programmed before and to people who _have_. If they have no preconceived idea of Booleans, then I do not propose to need to teach it to them in an introductory lesson. There is a time to learn symbolic logic but while trying to learn how to program for the first time is not it. If, on the other hand, they _do_ have some preconceived idea of Booleans then Python will not have what they want. They will want stricter booleans that 'behave properly'. Maybe some day we should give these people such a type. People who do symbolic logic and make push-down automatons all day long will love it. If we implement this, not out of objects, but out of bits, and have something really sparse in memory consumption, we will cause jubilation in the NumPy community as well. But I don't want to discuss this here, I only want to discuss the new type which is proposed in this PEP. And I believe that this half-way thing, int-in-hat, that is called bool but which does not implement what a mathematician would call truth values will make it much harder for me to teach the Python language and programming in general. Not only do I not need, but I actively do not want this change which is the integer 0 and the integer 1 that have some odd printing properties. 1) Should this PEP be accepted at all. No. 2) Should str(True) return "True" or "1": "1" might reduce backwards compatibility problems, but looks strange to me. (repr(True) would always return "True".) Now we have a teaching problem. If str(True) returns anything but 'True' then I am going to have to explain this to newbies, really early on. I can't see myself claiming that 1 is the string representation of True. I can see myself explaining that int(True) is 1, or that bool(1) is True. If I say that the string representation of True is 1, then I must assert that True is just a fancier, prettier way of writing 1. But this breaks the more common practice where str is used for the prettier way of writing things and repr is used for the uglier one. And I guarantee my students will notice this, especially if they have heard me explain why their floating point numbers are not printed the way that they expect. I don't want to have to explain that eval(repr(object)) is supposed to generate the object whenever possible, for the last thing I want newbies to be thinking about is eval(). I guess I am going to have to say that it is a wart on the language, and that we have it this way so as not to break too much exisiting code. I think this wart is far uglier than the lack of a half-way-but-not- quite Boolean Truth value. But I am all for this wart rather than break the exisiting code. 3) Should the constants be called 'True' and 'False' (corresponding to None) or 'true' and 'false' (as in C++, Java and C99). I would rather the constants, if we have them, be called anything other than True or False. It is not the things that you don't know that hurt you the most in learning a language -- it is 'the things that you know that ain't so'. Learning C, from PL/1, my first problem was 'how do I write a procedure'? You see, I aleady _knew_ that _by definition_ functions returned values and procedures didn't, and so, since I didn't want to return a value, I didn't want to write a function. Learning that your basic defintions are wrong generally requires a poke from the outside. The same thing happened to me when learning Python. I knew, by definition, that attributes were not callable. Using getattr() to test whether a class had a method did not occur to me, despite pouring over python docs for 2 days looking for such a thing. Finally I went after it out of the __class__ -- and immediately posted something to python-list, saying 'This is ugly as sin. What do real people do?' I never got around to questioning whether 'attributes by definition, may not be a callable' in precisely the same way I never got around to questioning whether 'you could write a function that did not return a value'. The problem is that _everybody_ has some conceptual understanding of True and False. And whatever that understanding is, it is unlikely to be the one used by Python. Python does not distinguish between True and False -- Python makes the distinction between something and nothing. And I think that this is one of the reasons why well-written Python programs are so elegant. And this is what I am trying to teach people. So I out-and-out tell people this. {} is a dictionary-shaped nothing. [] is a list-shaped nothing. 0 is an integer-shaped nothing. 0.0 is a float shaped nothing. I want to save them from the error of writing if (bool(myDict) == True): and if they start out believing that python only distinguishes between Something and Nothing, they mostly are ok. And to rub this point in you can do this: >>> False = 1 >>> True = 0 >>> if False: ... print 'Surprise!' ... Surprise! >>> if True: ... print 'I am True' ... else: ... print 'Surprise Again!' ... Surprise Again! This is a very nice eye-opener. It is a true joy. Watch the minds go *pop*! and the preconceived notions disappear. (You then let them know exactly what you will think of them if they ever do this 'for real', of course.) 4) Should we strive to eliminate non-Boolean operations on bools in the future, through suitable warnings, so that e.g. True+1 would eventually (e.g. in Python 3000 be illegal). Personally, I think we shouldn't; 28+isleap(y) seems totally reasonable to me. This is not an argument for allowing non-Boolean operations on bools(); this is an argument for not writing functions that return Booleans. Make them return numbers instead, so that you can use them as you did. Last month we discussed why in 1712 February had 30 days in Sweden. (See: http://groups.google.com/groups?q=leap+year+Sweden+group: comp.lang.python.*&hl=en&selm=Xns91D08815184B2svenaxelssonbokochwe% 40212.37.1.234&rnum=1 if you care, and you missed it.) I live in Sweden. Assigning students the problem of calculating whether or not a given year is a leap year in Sweden appeals to me. But I know students. I guarantee that I will get an isleap that returns a True or a False under the proposed new regime. And this is precisely what I do not want, which I will try to teach by assigning, next week, a program that calculates how many days are in a given year. I predict that I will get a lot of answers like this: if year == 1712: days = 367 elif isSwedishLeap(year): days = 366 else: days = 365 There are many things wrong with this code. The bizzare special case of 1712 belongs in the SwedishLeap function, with the rest of the weirdnesses. Thus I will have to convince my students that it is better to write a function that does not return True or False. And this is despite the fact that I originally asked for 'is Y a leap year or not'. This will be a necessary exercise. If True and False are in the language, I am going to have to work especially hard to teach my students that you (mostly) shouldn't use the fool things. There is almost always some value that you would like to return instead. Having renamed SwedishLeap and fixed it to return how many leap days, instead of a bool, I have now made a different problem for myself and my students. The new improved solution for 'is this year a leap year' will be: if bool(SwedishLeap(year)) == True: # the better students will say 'is' instead of '==' print 'yes.' else: print 'no.' Aaargh! I already see too much code like this. It's mostly written by people who come from other languages. They define their own True and False so they can do this. (And they mostly have an extra set of () as well). Right now I have the perfect fix for this. I just say 'Python does not care about True and False, only nothing or something'. You have just stolen my great weapon. What am I going to say? attempt 1. Python pretends to have bools, but they are just ints in fancy hats. So you are making more comparisons than are necessary. smart student: But you said it is better to be explicit than implicit! And here I am explicitly performing the type coercion rather than let it happen implicitly! (or PyChecker _warned_ me that I was making an implicit conversion!) I put in the cast so that people will know exactly what is happening! attempt 2. But they are _really_ ints 'under the hood'. I was not kidding about the fancy hats! if SwedishLeap(year): is precisely what you want. You don't want to test against True at all! smart student: Comparisons yield boolean values. Therefore they _want_ a Boolean value. You are just being lazy because it saves typing. In the bad old days before we had Booleans this was ok, but now that we have them we should use them! Otherwise what good are they? What should I be using them for if not for this? attempt 3. Got an hour? I'd like to explain signature-based polymorphism to you ... smart student: ha! ha! ha! 5) Should operator.truth(x) return an int or a bool. Tim Peters believes it should return an int because it's been documented as such. I think it should return a bool; most other standard predicates (e.g. issubtype()) have also been documented as returning 0 or 1, and it's obvious that we want to change those to return a bool. I think that operator.truth(x) should return an int because about the only thing I use it for is operator.truth(myDict). I really want the integer value. How do you propose I get it if you change things? Why make me go through the gyrations of int(bool(myDict))? By the way, look at the list of those things that would be changed to return a bool. Most of them are python implementations of ANCIENT C functions. They date from the time before we invented exceptions! The primitive old days when we had to test for every possible time you wouldn't want to run your code before you actually got to run it. I don't want to go back to the days of if aflag or (bflag and (cflag or dflag)): either. And all Truth testing (unless you are doing symbolic logic) reeks this way to me, of the old style I am trying to stamp out. Truth testing is just another form of type testing, and just as ugly. Rationale Most languages eventually grow a Boolean type; even C99 (the new and improved C standard, not yet widely adopted) has one. Many programmers apparently feel the need for a Boolean type; most Python documentation contains a bit of an apology for the absence of a Boolean type. So fix the docs, don't change the code! . I think the fact that in python control flow structures distinguish between Something and Nothing is one of the beauties and glories of the language, and you should delete any documentation that says otherwise. Under the proposed new scheme you will have to trade apologies for the lack of bools, for apologies for not producing real bools, only this int-in-a-new-hat hack that pretends to be a bool. This is hardly progress. I've seen lots of modules that defined constants "False=0" and "True=1" (or similar) at the top and used those. The problem with this is that everybody does it differently. For example, should you use "FALSE", "false", "False", "F" or even "f"? And should false be the value zero or None, or perhaps a truth value of a different type that will print as "true" or "false"? Adding a standard bool type to the language resolves those issues. So would adding True and False to the __builtins__, and probably operator.truth as well, and then modifying PEP 8 saying to use the things if you actually have a need for True and False. Then you could also get a much needed word in edgewise discouraging if bool(x) == True: or actually using True and False much, because there is usually a better more pythonic way to do what people used to other languages are accustomed to doing with booleans. This is precisely what some people have said here: 'When I started using Python, I made True and False, but once I stopped trying to program in some other language using python, I stopped needing these things'. (see the recent post by Don Garrett in this thread for an example.) This is what I have observed as well. And I fear if you add these new types to the langauge people will never take this step. The existence of the types in the language will discourage them from thinking that using True and False all the time is not pythonic. It is nice that people are puzzled, wondering how all those python programmers live without a boolean type. Eventually they puzzle it out. This is not a bug, but a feature . Some external libraries (like databases and RPC packages) need to be able to distinguish between Boolean and integral values, and while it's usually possible to craft a solution, it would be easier if the language offered a standard Boolean type. I'm one of the people who build interfaces to databases that need to distinguish this. For what it's worth, can you please not add this feature to the language? Don't do it for me .... The standard bool type can also serve as a way to force a value to be interpreted as a Boolean, which can be used to normalize Boolean values. Writing bool(x) is much clearer than "not not x" and much more concise than if x: return 1 else: return 0 Conciseness for its own sake is no virtue. Here are some arguments derived from teaching Python. When showing people comparison operators etc. in the interactive shell, I think this is a bit ugly: >>> a = 13 >>> b = 12 >>> a > b 1 If this was: >>> a > b True it would require one millisecond less thinking each time a 0 or 1 was printed. This is the basis of our disagreement. I think that it is very, very important that much more than a millisecond be spent on this. This is a fundamental python concept, which I want to teach. This is precisely where you learn that python distinguishes between Something and Nothing, and if you have a problem seeing why this implies a > b printing as 1, then you probably have a problem with the whole concept. And making it return True is precisely what I never, ever, ever want a python learner to see. People who come from staticly declared languages have a terrible burden to overcome when they meet python. They are not used to the fact that everything is an object. They have rigid barriers in their heads between 'control statements' and 'data'. This is precisely where such conceptual barriers first begin to crumble. This is precisely the experience I _want_ my students to have. It is precisely how I train people to give up their old ideas. You have just made my teaching job harder, not easier. I've never had any trouble teaching anybody that if 1: means do it, and if 0: means don't. Ever. Who is it that has had such trouble? I fear they may be new to teaching and are confusing 'this is taking the students a while to learn because it is something new that they have never seen before' with 'this is taking the students a while to learn because the language is broken'. Learning that python distinguishes between Something and Nothing is a completely new idea that you are giving these people, outside of their experience. This is going to take a while to get used to. But it is going to take _longer_ to get used to, if there is all this confusing stuff about symbolic logic, truth values, George Boole, and why the math majors in the class are all snickering and saying 'Python is a sucky language. Its implementation of booleans is so, so, lame...' thrown in as well. If you already know what a boolean is, then chances are Python's bools are not going to behave the way you expect them to. If you don't know what a bool is, then I feel morally obligated to teach you the difference between real truth values and this int-in-a-hat kludge. In either case, I now have to spend a bunch of time teaching about booleans, something I had no desire to do before today. I _want_ to teach that if 1: means do it and if 0: means don't. I want to teach that python makes a distinction between Something and Nothing. And I can teach that a _lot_ faster than An Introduction to Boolean Algebra ... There's also the issue (which I've seen puzzling even experienced Pythonistas who had been away from the language for a while) that if you see: >>> cmp(a, b) 1 >>> cmp(a, a) 0 you might be tempted to believe that cmp() also returned a truth value. If ints are not (normally) used for Booleans results, this would stand out much more clearly as something completely different. I don't think that people are confused about this because they think that cmp is returning True or False. I've had people go on believing that cmp should return one of 2 values even as I was telling them that it returned one of 3. The desire for cmp to be a two-valued comparison runs deep in many souls, and is tied to a passionate belief that comparisons are binary, not that they return True and False. I don't see this as a problem, and I don't want evolve the language in this direction either; I don't believe that a stricter interpretation of "Booleanness" makes the language any clearer. I think that a really strict boolean might be nice to have. And if you ever write one, you will curse the day you let this hack be named bool. Replacing this int-with-a-hat with real bools will break so much code. Other languages (C99, C++, Java) name the constants "false" and "true", in all lowercase. In Python, I prefer to stick with the example set by the existing built-in constants, which all use CapitalizedWords: None, Ellipsis, NotImplemented (as well as all built-in exceptions). Python's built-in module uses all lowercase for functions and types only. But I'm willing to consider the lowercase alternatives if enough people think it looks better. I'd really like something other than True and False altogether. Something Nothing or Empty Full or Yin Yang, __anything__ which people will not believe they already understand. It is so much easier to teach something which people know is brand new rather than something which is brand new but looks like something subtly different that people already believe they know. I have this problem a lot. It is hard to teach people what floating point numbers are because their grade school teachers have taught them what fixed point decimals are so well (except for the name). If floating point numbers were traditionally printed xxx#yyy instead of xxx.yyy I do not think that I would have such difficulties. In any case the people who do not understand floating point would be aware that there is something that they do not understand, rather than blinding going off an using them to represent money. I don't want to have to teach the pythonic meaning of True and False to people who already believe they know what True and False is. This is going to be hell. It is about as hard a thing as there is in teaching, getting around people's previous conceptions. It has been suggested that, in order to satisfy user expectations, for every x that is considered true in a Boolean context, the expression x == True should be true, and likewise if x is considered false, x == False should be true. This is of course impossible; it would mean that e.g. 6 == True and 7 == True, from which one could infer 6 == 7. Similarly, [] == False == None would be true, and one could infer [] == None, which is not the case. I'm not sure where this suggestion came from; it was made several times during the first review period. For truth testing of a value, one should use "if", e.g. "if x: print 'Yes'", not comparison to a truth value; "if x == True: print 'Yes'" is not only wrong, it is also strangely redundant. This suggestion came from somebody's preconceived idea of 'what is True' and 'what does it mean for something to be True'. Now that you have posted this to python-list, you have found a whole lot more. Have you ever had such a response to a PEP? Everybody thinks that they know what True and False means, and that Python should do it his or her own way. This is what everybody's classroom looks like as well. And this is why sane teachers do not want to discuss the meaning of True if they can help it. I want to keep python out of the True and False business. Python cares about whether a value is Something or Nothing. This is beautiful, and _better_ than what the other languages do. Once again, Thank you for reading this and giving me a chance to write up my objections. You have made me a better teacher because of this. Laura Creighton From guido@python.org Tue Apr 2 13:57:29 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 02 Apr 2002 08:57:29 -0500 Subject: [Python-Dev] Re: PEP 279 In-Reply-To: Your message of "Tue, 02 Apr 2002 00:28:45 EST." <002d01c1da07$434c88c0$4c66accf@othello> References: <002d01c1da07$434c88c0$4c66accf@othello> Message-ID: <200204021357.g32DvTj06129@pcp742651pcs.reston01.va.comcast.net> > Summary, I think we're down to subtle differences and have the following > preference order: > 1. enumerate Forget the others you mentioned, but insert one at the top, so we get: 0. itemize 1. enumerate --Guido van Rossum (home page: http://www.python.org/~guido/) From python@rcn.com Tue Apr 2 14:07:28 2002 From: python@rcn.com (Raymond Hettinger) Date: Tue, 2 Apr 2002 09:07:28 -0500 Subject: [Python-Dev] Re: PEP 279 References: <002d01c1da07$434c88c0$4c66accf@othello> <200204021357.g32DvTj06129@pcp742651pcs.reston01.va.comcast.net> Message-ID: <005b01c1da4f$b9b67aa0$4a66accf@othello> > Forget the others you mentioned, but insert one at the top, so we get: > > 0. itemize > 1. enumerate Itemize() is too close to the unrelated concept of items: >>> d = { 'red':'roja', 'blue':'azul' } >>> d.items() [('blue', 'azul'), ('red', 'roja')] >>> list(itemize(d)) [(0,'blue'), (1,'red')] Let's stick with enumerate(). Raymond From guido@python.org Tue Apr 2 14:27:23 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 02 Apr 2002 09:27:23 -0500 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Misc NEWS,1.146.2.9,1.146.2.10 In-Reply-To: Your message of "Tue, 02 Apr 2002 16:06:20 +1000." <200204020606.g3266Kd01556@mbuna.arbhome.com.au> References: <200204020606.g3266Kd01556@mbuna.arbhome.com.au> Message-ID: <200204021427.g32ERNY06198@pcp742651pcs.reston01.va.comcast.net> [Moved from the checkins mail] > > + What's New in Python 2.1.3? > > + Release date: XX-XXX-XXXX > > + =========================== > > + > > + Core > > + > > + - Fixed two bugs reported as SF #535905: under certain conditions, > > + deallocating a deeply nested structure could cause a segfault in the > > + garbage collector, due to interaction with the "trashcan" code; > > + access to the current frame during destruction of a local variable > > + could access a pointer to freed memory. > > + > > + [Anthony] > Given the impact on Zope, I'd say this probably justifies a 2.1.3 > release. I'm ok to do it, although I don't think it's worth doing > the back-trawl through the change logs to find other suitable > patches. I don't recall any other bugs ugly enough to merit > inclusion, but I'm open to suggestions :) If you want to do it, that's great! AFAIC you can release exactly what's in the release21-maint branch now -- only update the README and LICENSE file. Fred made some checkins to the docs, I'm not sure if that means he wants to make a 2.1.3 doc release too (personally, I'm against spending more time on it than absolutely necessary). There's one issue: I'd rather not see Tim waste time on recreating the Windows setup he used for the 2.1.2 installer. Maybe this can be a source-only release? That's good enough for Zope (while Zope technically supports Windows, nobody uses it this way, certainly not the people who are likely to be bit by the above problem). --Guido van Rossum (home page: http://www.python.org/~guido/) From dhl660717@lycos.co.kr Tue Apr 2 13:54:46 2002 From: dhl660717@lycos.co.kr (Áø³ª¶ó) Date: Tue, 2 Apr 2002 22:54:46 +0900 Subject: [Python-Dev] NEW ¹Ù´Ï(Barney) ºñµð¿À SET (±¤..°í) Message-ID:
 
  ¢º ±¸¼º : ºñµð¿À 3Æí (¿µ¾îÀÚ¸·) + ¿µ¾î´ëº»Áý 3°³ ¢¸

À̹ø¿¡ ¼Ò°³ÇÏ´Â »õ·Î³ª¿Â ½ÅÀÛ Barney and FriendsÀÇ ¼¼°¡Áö ¿¡ÇǼҵå´Â ¹Ù´ÏÀÇ ÆÒµéÀ̶ó¸é ÀÌ¹Ì ¾Æ¸¶Á¸¿¡¼­ ±¸ÀÔÇϽñ⸦ ¸î¹ø ½ÃµµÇϼ̴ø °æÇèÀ» °¡Áö°í °è½Ç°Ì´Ï´Ù.

ù¹ø° ¹Ù´Ï ºñµð¿À ÄÝ·º¼ÇÀº ÀÌ¹Ì ÀúÈñ »çÀÌÆ®¿¡¼­µµ ¿¹Àü¿¡ ¼Ò°³µå¸°¹Ù ÀÖÀ¸¸ç KBS 2TV¿¡¼­ ´Ù½Ã ¿ì¸® ³ª¶ó ¾ÆÀ̵é·Î Àç Á¦ÀÛÀÌµÈ Çѱ¹¾îÆÇ ¹æ¼ÛÀÌ ³ª°¡°í ÀÖÀ» Á¤µµ·Î ÀÌ¹Ì ±× ±³À°ÀûÀÎ È¿°ú¸¦ ÀÎÁ¤¹ÞÀº ÀÛÇ°À̱⠶§¹®¿¡ ÀÚ½ÅÀÖ°Ô ±ÇÇØ µå¸±¼ö ÀÖ½À´Ï´Ù.

¡Ú¡Ú ¹Ù´Ï ºñµð¿À ÀÎÁõ ³»¿ª

Ãâ½ÃÀÌÈÄ 6õ5¹é¸¸°³ ÀÌ»óÀÇ ÆÇ¸Å°í ±â·Ï
Ãâ½ÃÀÌÈÄ 10³â°£ ¹Ì±¹ ¾î¸°ÀÌ Box office 1À§
¹Ù´Ï ij¸¯ÅÍ 1999³â ¹Ì±¹³» ÀÎÁöµµ Á¶»ç 1À§ Â÷Áö
¹Ì±¹ ºÎ¸ðµéÀÌ »ÌÀº ±× ÇØÀÇ ÃâÆÇ»ó ¼ö»ó
TVÀÇ ¾ÆÄ«µ¥¹Ì»ó ¿¡¹Ì»ó ¼ö»ó
¹Ì±¹ PBS¹æ¼Û±¹À» ÅëÇØ 10³â °£ ¹æ¿µ
ÇöÀç AFNµîÀ» ÅëÇØ Àü ¼¼°èÀûÀ¸·Î ÀýÂù¸® ¹æ¿µ Áß
¹Ù´Ï ¿Àµð¿À ¹é¸¸Àå ÀÌ»óÀÇ ÆǸŷ®À¸·Î Ç÷¡Æ¼³Ñ ȹµæ
¿¹ÀÏ´ë ¿¬±¸°á°ú ¹ÌÃëÇÐ ¾î¸°À̸¦ À§ÇÑ ÃÖ°íÀÇ ÇÁ·Î±×·¥À¸·Î ¼±Á¤


±×·¯¸é ÀÌ·¯ÇÑ ¹Ù´ÏÀÇ ±³À°ÀûÀÎ È¿°ú¿¡ ´ëÇؼ­ ÀÚ¼¼È÷ ¾Ë¾Æº¸°Ú½À´Ï´Ù.
¿ì¼± ù°·Î, '¹Ù´Ï¿Í Ä£±¸µé'¿¡¼­´Â ¾î¸°À̵éÀÌ »ó»óÇÏ´Â ¸ðµç °ÍÀ» ÀÌ·ê ¼ö ÀÖ°Ô ÇØ ÁÜÀ¸·Î½á ¾î¸°À̵éÀÌ ´Ù¾çÇÏ°í »ö´Ù¸¥ »ý°¢µé°ú ¿¹»Û ²ÞµéÀ» Å°¿ï ¼ö ÀÖµµ·Ï ÇÏ°í ÀÖ½À´Ï´Ù.


»ó»óÀÇ ÁÖÁ¦´Â ¾î¸°À̵éÀÌ ÇÏ°í ½Í¾îÇÏ´Â ÀÏ, °í¹ÎÇÏ°í ÀÖ´Â ÀÏ, °¡°í ½ÍÀº °÷ µî...¾î¸°À̵éÀÌ ¿øÇÏ´Â ¸ðµç °ÍÀ» ´ã¾Æ³»°í ÀÖ½À´Ï´Ù. ¾î·È´ø ½ÃÀý ´©±¸³ª ÇغÃÀ»¸¸ÇÑ ±Í¿©¿î »ó»óµé, ³»¸®´Â ºñ°¡ ·¹¸ó»çÅÁ°ú Á©¸®¶ó¸é? ȤÀº ¹ÐÅ© ½¦ÀÌÅ©¶ó¸é? Á¤¸» »ó»ó¸¸ Çصµ Áñ°ÌÁö¿ä ÀÌ·¯ÇÑ Áñ°Å¿òÀ» ¹Ù´Ï´Â ¾ÆÀ̵鿡°Ô ½Å³ª°Ô ³ë·¡ÇØ ÁÝ´Ï´Ù.

µÎ¹ø°·Î,¹Ù´Ï¿¡¼­ °¡Àå Áñ°Å¿î ½Ã°£Àº ³ë·¡¿Í ÃãÀÌ ÀÖ´Â ½Ã°£ÀÌ ºÐ¸íÇÕ´Ï´Ù.
ºñµð¿À ÇÑÆí¿¡ ´ëºÎºÐ 10°îÀÌ Èǽ ³Ñ°Ô ³ë·¡°¡ ¸¹ÀÌ ³ª¿À´Âµ¥, ÀÌ ³ë·¡µéÀº ¾î¸°À̵éÀÌ ¹Ù¶ó´Â ¼¼°è¿Í Èñ¸ÁÀ» ¸»Çϱ⵵ ÇÏ°í, ³»¿ëÀ» º¸´Ù ½±°í, Àç¹ÌÀÖ°Ô ¹è¿ï ¼ö ÀÖµµ·Ï µµ¿ÍÁÖ´Â ¿ªÇÒÀ» ÇØÁÝ´Ï´Ù.

¶ÇÇÑ ¸¶Áö¸·À¸·Î ¹Ù´Ï´Â 'ºñµð¿À¸¦ º¸¿©ÁÖ´Â ºÎ¸ð'·Î½á °í¹ÎÇÒ ¼ö ÀÖ´Â ºÎºÐÀÎ '¾ÆÀ̵éÀÇ °í¸¥ ¹ß´Þ'À» µµ¿ÍÁÖµµ·Ï Á¦À۵Ǿú½À´Ï´Ù. ¹Ù´Ï°¡ °­Á¶ÇÏ´Â °ÍµéÀº- ¼­·Î »ç¶ûÇϱâ, ÇÔ²² ³ª´©±â, Á¤¸® Á¤µ·Çϱâ, »ó»óÇϱâ, ³²À» µµ¿ÍÁÖ±â, Áú¼­ÁöÅ°±â, ÇÏ´Ù¸øÇØ ±×¸² ±×¸°ÈÄ¿¡´Â ¼ÕÀ» ±ú²ýÀÌ ¾Ä±â±îÁö - ¾ÆÀÌ¿¡°Ô ¾î´À ÇÑ°¡Áö°¡ ¾Æ´Ï¶ó ¾ÆÀÌÀÇ °¨¼º°ú »çȸÀûÀÎ °ü°è, ½ÅüÀûÀÎ ¿µ¿ª°ú ±³À°ÀûÀÎ
³»¿ë±îÁö ±îÁö °í·ç µµ¿òÀ» ÁֹǷΠ±× È¿°ú°¡ º¸ÀÌÁö ¾Ê°Ô Å©´Ù°í ¸»ÇÒ ¼ö ÀÖ°Ú½À´Ï´Ù.

ƯÈ÷³ª À̹ø¿¡ÁغñÇÑ ¼¼ÆíÀÇ ¹Ù´Ï´Â Áñ°Å¿òµµ ¹°·ÐÀÌ°í, ÀÌÁ¦ ÀÔÇÐÀ» ¾ÕµÐ ºÎ¸ð´ÔµéÀ» °í·ÁÇÏ¿© ¹Ù´Ï ½Ã¸®ÁîÁß¿¡¼­µµ ±³À°ÀûÀÎ ³»¿ë¿¡ ÁßÁ¡À» µÐ ºñµð¿À¸¦ ¾ö¼±ÇÏ¿´±â ¶§¹®¿¡ ƯÈ÷ ±³À°ÀûÀÎ È¿°ú°¡ ´õ ¹è°¡ µÈ´Ù°í ÇÏ°Ú½À´Ï´Ù.





  Barney and Friends ij¸¯ÅÍ ¼Ò°³
 Barney´Â º¸¶ó»ö °ø·æÀÎÇüÀÌÀÚ ¾ÆÀ̵éÀÇ »ó»ó¼ÓÀÇ Ä£±¸ÀÔ´Ï´Ù.
Æò¼Ò¿¡´Â Á¶±×¸¸ ÀÎÇüÀÇ ¸ð½ÀÀÌÁö¸¸ ¾ÆÀ̵éÀÌ »ó»ó¸¸ Çϸé Å°°¡ ¹«·Á 180À̳ª µÇ´Â °ø·æÀ¸·Î º¯ÇÏ¿© ¾ÆÀ̵éÀÌ ¿øÇϴ°ÍÀ» Çö½Ç·Î ÀÌ·ç¾îÁÝ´Ï´Ù.
¹Ù´Ï´Â ´Ã ¹«ÇÑÇÑ »ó»ó·ÂÀ» °­Á¶ÇÕ´Ï´Ù.
¶ÇÇÑ ¾ðÁ¦³ª ¾ÆÀ̵éÀÇ ÁÁÀº Ä£±¸µµ µÇ¾îÁÖ¸ç Áñ°Å¿î ³îÀ̳ª Ãã, ³ë·¡¿Í °ÔÀÓÀ» º¸¿©ÁÖ¸ç Áñ°Å¿î À̾߱⸦ µé·ÁÁÝ´Ï´Ù.
¾ðÁ¦³ª ÈòÀ̸¦ º¸ÀÌ¸ç ½Ì±ß ¿ô´Â ¹Ù´Ï´Â±Ý¹æÀÌ¶óµµ ÅÍÁ®³ª¿ÃµíÇÑ Áñ°Ì°í ¿ôÀ½ °¡µæÇÑ ¸ñ¼Ò¸®·Î ¾ÆÀ̵鿡°Ô ²Þ°ú ȯ»óÀÇ ¼¼°è¸¦ º¸¿©ÁÝ´Ï´Ù.
¾ÆÀ̵éÀÇ Ä£±¸·Î´Â À̺¸´Ù ´õ ÁÁÀº Ä£±¸´Â ¾øÀ»¸¸Ä¡ ÈǸ¢ÇÏ°í Ä£ÀýÇÕ´Ï´Ù.
BJ´Â ¶Ù¾î³î±â ÁÁ¾ÆÇÏ´Â 7»ìÀÇ È°µ¿ÀûÀÎ °ø·æÄ£±¸ÀÔ´Ï´Ù.
¿îµ¿Àº ¹«¾ùÀ̵çÁö ÁÁ¾ÆÇÏÁö¸¸, ³ó±¸³ª ¾ß±¸µµ ÁÁ¾ÆÇÕ´Ï´Ù.
±×·¡¼­ ¾ðÁ¦³ª ¿îµ¿¸ðÀÚ(ÀÚ¼¼È÷ º¸¸é ¸Ó¸®¿¡ Àå½Äó·³ ¾ñÇô Àִ°ÍÀÌ ¿îµ¿ ¸ðÀÚÀÔ´Ï´Ù)¸¦ ¾²°í ÀÖ°í Ä¿´Ù¶õ ¿îµ¿È­¸¦ ½Å°í ÀÖ½À´Ï´Ù.
BJ´Â ¾î¸°À̵éÀ» ´ëÇ¥Çϴ ǥº»À̶ó ÇÒ¸¸Å­ ¾ÆÀ̵éÀÇ Á¤¼­¸¦ Àß º¸¿©Áִ ij¸¯ÅÍÀÔ´Ï´Ù.
¿©µ¿»ýÀÎ º£À̺ñ ¹ä°ú ¹°°Ç Çϳª¸¦ °¡Áö°í ¹Ð°í ¶¯±â¸ç ½Î¿ì´Â ¸ð½Àµµ ±×·¸Áö¸¸, °ð ¾çº¸ÇÏ°í ¼­·Î »çÀÌÁÁ°Ô Áö³»´Â ¸ð½ÀÀ» º¸¸é ¸¶Ä¡ ¿ì¸®ÀÇ ¾ÆÀ̵éÀÇ ¸ð½ÀÀ» º¸´Â ±âºÐÀÌ µì´Ï´Ù.
¸ñ¼Ò¸®´Â °³±¸ÀåÀÌ ¸ñ¼Ò¸®Ã³·³ Ç㽺ŰÇÏ°í ½ÅÀÌ ³³´Ï´Ù.
Baby BopÀº BJÀÇ ¿©µ¿»ýÀÌÀÚ BarneyÀÇ Ä£ÇÑ Ä£±¸·Î 3»ìÀÔ´Ï´Ù.
BJ´Â º£À̺ñ ¹äÀ» Sissy(sisterÀÇ ¾ÖĪ)¶ó°í ºÎ¸¨´Ï´Ù.
¾ðÁ¦³ª ³ë¶õ ´ã¿ä¸¦ ³¢°í ´Ù´Ï´Â ÀüÇüÀûÀÎ 3»ì ¾ÆÀÌÀÇ ¸ð½À°ú ÇൿÀ» º¸¿©ÁÝ´Ï´Ù.
³î´Ù°¡µµ Àáµé±âµµ ÇÏ´Â Àá²Ù·¯±â º£À̺ñ ¹äÀº ¾ÆÀ̵éÀÌ µ¿Áú°¨À» ´À³¥¼ö Àִ»ç¶û½º·¯¿î ij¸¯ÅÍÀÔ´Ï´Ù.
±×·± Á¡µµ º£À̺ñ ¹äÀ» º¸¸é¼­ ¾ÆÀ̵éÀÌ Áñ°Å¿ö ÇÏ°í º£À̺ñ ¹äÀ» Ä£±¸Ã³·³ µ¿»ýó·³ »ç¶ûÇÏ´Â ÀÌÀ¯ÁßÀÇ ÇϳªÀÏ°Í °°³×¿ä.
±Í¿©¿î ¸ñ¼Ò¸®¿Í Á¤¼­·Î ¾ÆÀ̵éÀÌ ¸Å·ÂÀ» ´À³¢´Â ij¸¯ÅÍÀÔ´Ï´Ù.

°ø·æÄ£±¸ ¿Ü¿¡µµ, ¸î¸íÀÇ ¾î¸° ¾ÆÀ̵éÀÌ Ä£±¸µé·Î ³ª¿É´Ï´Ù.
´Ùµé Á¤¸» Ãã°ú ³ë·¡¿¡ ´ÉÅëÇÑ Ä£±¸µé·Î½á, ¹Ýµå½Ã Ã⿬ÇÏ´Â ¾ÆÀ̵éÀÇ ÀÎÁ¾Àº ¹éÀÎ,µ¿¾çÀÎ,ÈæÀÎµî °ñ°í·ç °®Ãß¾îÁ® ÀÖ´Â °ÍÀÌ ´«¿¡ ¶ë´Ï´Ù.
±×°ÍÀº ¾î¼´Ù°¡ ±×·±°ÍÀÌ ¾Æ´Ï¶ó, º¸ÀÌÁö ¾Ê´Â°÷¿¡¼­µµ Ä¡¹ÐÇÏ°Ô ³ë·ÂÀ» ÇÑ ÈçÀûÀ¸·Î º¸ÀÔ´Ï´Ù.
±×·¸Áö ¾Ê°í ÈçÈ÷º¼ ¼ö ÀÖ´Â ¾ÆÀ̵éÀÇ ¾Ö´Ï¸ÞÀ̼ÇÀ» º¸´Ùº¸¸é ¹®µæ ¹éÀÎÀ̰ųª ¿¹»Û ¾ÆÀÌ°¡ ÂøÇÑ »ç¶÷À̰ųª ÁÖÀΰøÀÌ°í, ±×·¸Áö ¾ÊÀº »ç¶÷Àº ´Ù ³ª»Ú´Ù°í ÀνÄÇϴ°ÍÀº ¾Æ´Ñ°¡ °ÆÁ¤ÀÌ µÉ ¶§°¡ ¸¹ÀÌ ÀÖ½À´Ï´Ù.
¹Ù´Ï´Â ±×·± ¸é¿¡¼­ ¾ÆÀ̵鿡°Ô Á¶±Ý ´õ ³ÐÀº ¼¼°è¸¦ ÀνĽÃÄÑ ÁÖ°í, Æí°ß¾ø´Â ¼¼°è°üÀ» °®µµ·Ï µµ¿Í ÁÙ¼ö ÀÖÀ» °ÍÀÔ´Ï´Ù.
ÀÌÁ¦ ¹Ù´Ï¿Í ÇÔ²² »ó»óÀÇ ³ª¶ó·Î ¶°³ª º¸½Ç±î¿ä?


  Barney Let's Play School (ABC& 123)




Running Time: 50 minutes

Çб³ ¼ö¾÷ÀÌ ºÎ·¯¿î Baby BopÀº ¾ÆÁ÷ ¾î·Á¼­ Çб³¿¡ °¥ ¼ö ¾ø´Ù´Â »ç½ÇÀÌ ½½ÇÁ°Ô »ý°¢ÀÌ µË´Ï´Ù.
Barney´Â »ó»óÀÇ Çб³¸¦ ¸¸µé¾î Baby BopÀÌ ¾ÆÀ̵é°ú ÇÔ²² Áñ°Å¿î Çб³ ³îÀ̸¦ Çϵµ·Ï µµ¿ÍÁÝ´Ï´Ù.
Ãâ¼®µµ ºÎ¸£°í, ¾ËÆĺª, ¼ýÀÚ, »ö±ò, µµÇüµîÀ» ÀÍÈ÷°í, Ã¥¿¡ ³ª¿À´Â »¡°£ µÎ°ÇÀ» ¿¬±ØÀ¸·Î ÇÏ°Ô µÇÁö¿ä.
ÇÏÁö¸¸ º£À̺ñ ¹äÀÇ °³¼ºÃ³·³ ³ë¶û ´ã¿ä¸¦ ¿¬»ó½ÃÅ°´Â(Little Red Riding Hood°¡ ¾Æ´Ñ) "Little Yellow"°¡ µÇ¾î º¾´Ï´Ù.
ÀÌ°ÍÀº »ç½Ç Á¶±Ý ¹«¼­¿î ³»¿ë(´Á´ë°¡ ÇҸӴϸ¦ Àâ¾Æ¸Ô´Â ³»¿ëÀ̶óµçÁö)À» ¾ÆÁÖ ÄÚ¹ÍÇÏ°í ¹«¼·Áö ¾Ê°Ô ó¸®ÇÏ¿´½À´Ï´Ù.
üÀ°½Ã°£, Á¡½É½Ã°£µµ º¸³»´Âµ¥ º£À̺ñ ¹äÀÌ ÁغñÇÑ Á¡½É ³»¿ë ¶ÇÇÑ º£À̺ñ ¹äó·³ ±ôÂïÇÏ°í »ç¶û½º·´½À´Ï´Ù.
º£À̺ñ ¹äÀ» À§ÇÑ ¾ÆÀ̵éÀÇ Çб³ ³îÀÌ, ÇÔ²² ³ª´©´Â Áñ°Å¿ò°ú µû¶æÇÔÀÌ º¸ÀÌ´Â ºñµð¿À ÀÔ´Ï´Ù.

¢º±×¸®°í, Ȥ½Ã³ª °®°í °è½ÅºÐÀ» À§ÇØ ¸»¾¸µå¸®´Âµ¥ ÀÌ Á¦Ç°Àº Let's Play School°ú °°Àº Á¦Ç°ÀÓÀ» ¹Ì¸® ¾Ë·Áµå¸³´Ï´Ù.

Song List:

Barney Theme Song
One, Two, Buckle My Shoe
Today, We Can Say!
Finger Play Medley:
The Itsy Bitsy Spider
Where Is Thumbkin?
The Alphabet Parade
I Love to Read!
Mix a Color
Mac & Cheese
Little Butterflies
The Shape Song
The Baby Bop Hop
My Yellow Blankey
Today, We Can Say! (Reprise)
I Love You


  Barney - Rhyme Time Rhythm





Running Time: 50 minutes

Rhyme Time RhythmÀº Á¦¸ñó·³ ¶óÀÓ°ú ¸¹Àº ³ë·¡µéÀÌ ³ª¿É´Ï´Ù.
¿¡¹Ð¸®´Â ¶óÀÓÀ» ¾²°í ½Í¾î ÇÏÁö¸¸ Àß µÇÁö ¾Ê½À´Ï´Ù.
À̶§ Ã¥¼Ó¿¡¼­ ³ª¿Â 'Mother Goose'°¡ ³ë·¡¸¦ ºÒ·¯Áָ鼭 ¶óÀÓÀÇ ¹¦¹Ì¸¦ ¾Ë·ÁÁÝ´Ï´Ù.
°¢Á¾ »ó»óÀÇ ³ª¶ó¿Í Ã¥¼ÓÀÇ À̾߱âµéÀ» µè°í ¸¹Àº ¸¶´õ ±¸Á ºÎ¸£´Â °¡¿îµ¥ ¿¡¹Ð¸®¿Í Ä£±¸µéÀº ¶óÀÓÀÇ Áñ°Å¿ò°ú ¿ä·ÉÀ» ¾Ë°Ô µË´Ï´Ù.
³ë·¡ ¸®½ºÆ®¸¦ º¸¸é ¾Ë°ÚÁö¸¸ ±²ÀåÈ÷ ¸¹Àº ³ë·¡µéÀÌ ¼Ò°³°¡ µË´Ï´Ù.
¹Ù´Ï°¡ âÀÛ°îÀÌ¶ó¼­ ¸Á¼³ÀÌ´ø ºÐÀÌ °è½Ã´Ù¸é ÀÌ Rhyme Time RhythmÀº À¯¸íÇÏ°í ±Í¿¡ Àͼ÷ÇÑ ¼ö¸¹Àº ¸¶´õ ±¸ÁîµéÀ» Áñ±æ ¼ö À־ ÁÁÀº ºñµð¿ÀÀÔ´Ï´Ù.

Song List:

Did You Ever See A Lassie?
Hey, Diddle, Diddle
I'm Mother Goose
Here Sits The Lord Mayor
Daffy-Down-Dilly
London Bridge
Peter Piper
Humpty Dumpty
The Clock
To Market
Donkey, Donkey, Old and Gray
Mother Goose Sheep Medley
Baa, Baa Black Sheep
Little Boy Blue
Little Bo Peep
Mary Had A Little Lamb
This Is The House That Jack Built
The Land of Mother Goose
Mary, Mary Quite Contrary
Goosey, Goosey Gander
Little Miss Muffet
Jack Be Nimble
Peter, Peter Pumpkin Eater
There Was An Old Woman Who Lived In A Shoe
A Tisket, A Tasket
There Was A Little Girl and She Had A Little Curl
Just Like Me
Tea Party Medley
Polly Put The Kettle On
Little Jack Horner
Do You Know The Muffin Man
Pease Porridge Hot
Hot Cross Buns, Hot Boiled Bean
The Queen of Hearts
Pat-A-Cake
Roses Are Red
I Love You


  Barney - Super Singing Circus




Running Time: approximately 50 minutes

¾ÆÀ̵éÀº Åä¿äÀÏ¿¡ ¼­Ä¿½º¸¦ ÇÏ·Á°í ÇÕ´Ï´Ù. ÇÏÁö¸¸ ¿¡¹Ð¸®°¡ Åä¿äÀÏ¿¡ ¼­Ä¿½º¸¦ ÇϱⰡ °ï¶õÇÕ´Ï´Ù.
¿¡¹Ð¸®µµ ´Ù°°ÀÌ ¼­Ä¿½º¿¡ Âü¿©ÇÏ°í ½Í¾îÇÏ´Â ¾ÆÀ̵éÀ» À§ÇØ ¹Ù´Ï°¡ ´çÀå ¼­Ä¿½º¸¦ ÇÒ¼ö ÀÖµµ·Ï µµ¿ÍÁÖÁö¿ä.
¼­Ä¿½º¿¡¼­ ÇÏ´Â ¸ðµç ¹¦±â¿Í Áñ°Å¿òÀ» º¸¿©ÁÖ´Â ¾ÆÀ̵é°ú ±×·¡µµ »çÀÌ»çÀÌ ÀÌ´Û±âÀÇ Á߿伺À̳ª °Ç°­ÇÔÀ» À§ÇÑ ¿îµ¿ÀÇ Á߿伺µîÀ» °­Á¶ÇØ ÁÝ´Ï´Ù.
³ë·¡ °î¼ö°¡ À§ÀÇ µÎÆí¿¡ ºñÇØ Á¶±Ý ÀûµíÀÌ, º¼°Å¸®µµ ¸¹¾Æ Àç¹ÌÀÖ°í ±× ¾Æ·¡¿¡´Â ¼­·Î µ½°í ¹è·ÁÇÏ´Â »ç¶ûÀ» º¸¿©ÁÖ´Â ºñµð¿ÀÀÔ´Ï´Ù.

Song List:

Animal Fair
Puttin' On A Show
When The Circus Comes To Town
Marching Song
The Rainbow Song
Laugh With Me
Boom Boom, Ain't It Great To Be Crazy?
Brushing My Teeth
The Squirrel On The Flying Trapeze
Yankee Doodle
The Exercise Song
Me And My Teddy
When The Circus Comes To Town (reprise)
I Love You

From barry@zope.com Tue Apr 2 14:29:26 2002 From: barry@zope.com (Barry A. Warsaw) Date: Tue, 2 Apr 2002 09:29:26 -0500 Subject: [Python-Dev] Re: PEP 279 References: <002d01c1da07$434c88c0$4c66accf@othello> <200204021357.g32DvTj06129@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15529.49222.425446.876764@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> Forget the others you mentioned, but insert one at the top, GvR> so we get: | 0. itemize | 1. enumerate itemize() is not bad! -Barry From guido@python.org Tue Apr 2 14:37:07 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 02 Apr 2002 09:37:07 -0500 Subject: [Python-Dev] Deprecating C APIs (Use of PyArg_NoArgs()) In-Reply-To: Your message of "Tue, 02 Apr 2002 11:18:55 +0200." <3CA9777F.BB3DA0DD@lemburg.com> References: <3C9F4AC0.C076679B@metaslash.com> <3C9F8B0F.D921E530@metaslash.com> <200203252052.g2PKqJa08509@pcp742651pcs.reston01.va.comcast.net> <3C9FA9D1.DA46BE04@metaslash.com> <200203252300.g2PN0DY09544@pcp742651pcs.reston01.va.comcast.net> <3CA9777F.BB3DA0DD@lemburg.com> Message-ID: <200204021437.g32Eb7C09097@pcp742651pcs.reston01.va.comcast.net> > +1 on removing PyArg_GetInt() OK. > -1 on removing PyArg_NoArgs() ... this API is still in very active > use in third party extensions. Would it be OK if it issued a warning? > I think we should come up with a better way to deprecate C level > APIs. NEWS is not as effective as one might wish; any deprecation > or removal of an API should also include upgrade information > for the C extension writer so that upgrading their modules > does not require diving into the new style C API code every > time. In this case, PyArg_ParseTuple(args, ":functionName") is pretty simple to explain. > I'd suggest to have a PEP which documents all C API changes > or deprecations together with a short decsription of how to > update old code. It shouldn't be a PEP (that's a *proposal*). It should be part of the C API docs, possibly an appendix. > Another aspect which we ought to consider is that C extension > writers may have a different view of when to leave backward > compatibility path in favour of a new Python version. IMHO, > there should always be a way to emulate old behaviour so that > extensions can continue to support Python 1.5.2 if they need to. Not a problem in this case. --Guido van Rossum (home page: http://www.python.org/~guido/) From neal@metaslash.com Tue Apr 2 14:35:18 2002 From: neal@metaslash.com (Neal Norwitz) Date: Tue, 02 Apr 2002 09:35:18 -0500 Subject: [Python-Dev] Deprecating C APIs (Use of PyArg_NoArgs()) References: <3C9F4AC0.C076679B@metaslash.com> <3C9F8B0F.D921E530@metaslash.com> <200203252052.g2PKqJa08509@pcp742651pcs.reston01.va.comcast.net> <3C9FA9D1.DA46BE04@metaslash.com> <200203252300.g2PN0DY09544@pcp742651pcs.reston01.va.comcast.net> <3CA9777F.BB3DA0DD@lemburg.com> Message-ID: <3CA9C1A6.CB16C1ED@metaslash.com> "M.-A. Lemburg" wrote: > I think we should come up with a better way to deprecate C level > APIs. NEWS is not as effective as one might wish; any deprecation > or removal of an API should also include upgrade information > for the C extension writer so that upgrading their modules > does not require diving into the new style C API code every > time. > > I'd suggest to have a PEP which documents all C API changes > or deprecations together with a short decsription of how to > update old code. I think a PEP will be ignored just as easily as NEWS. Perhaps, we could wrap all deprecated functions/macros in: #ifdef Py_DEPRECATED #endif So if an extension wants to use deprecated functions, they would need to #define Py_DEPRECATED. We could also use another macro Py_OBSOLETE. In this way functions/macros would move into Py_DEPRECATED, then Py_OBSOLETE, then removed. The functions would still be in a library. And when a function is used a warning would be produced since there is no prototype. Not sure how to handle macros, should they remain undefined and create errors? If we wanted to be more heavy handed, we could create errors that would force extensions to be compiled w/Py_DEPRECATED by doing: #ifdef Py_DEPRECATED #define Py_MACRO(a) do_something(a) int Py_prototype(...); #else /* Create a syntax error unless Py_DEPRECATED is defined */ #define Py_MACRO(a) === #define Py_prototype === #endif > Another aspect which we ought to consider is that C extension > writers may have a different view of when to leave backward > compatibility path in favour of a new Python version. IMHO, > there should always be a way to emulate old behaviour so that > extensions can continue to support Python 1.5.2 if they need to. Perhaps, we could put the deprecated functions into a libpythoncompat.a. It would probably be best to move the functions out of the code they are in and into a new compat.c module. I'm not sure how workable this idea is. Neal From David Abrahams" <004b01c1d9fd$8be15c00$0202a8c0@boostconsulting.com> <200204021336.g32Dakb05964@pcp742651pcs.reston01.va.comcast.net> Message-ID: <016a01c1da56$216033c0$0202a8c0@boostconsulting.com> ----- Original Message ----- From: "Guido van Rossum" > [David Abrahams] > > I missed what this function does, > > x(seq) returns [(0, seq[0]), (1, seq[1]), ...] Oh. Not sure why anyone wants to build that into the language, but enumerate() seems like a decent name for it. itemize() seems OK too. just-call-me-wafflebird-ly y'rs, Dave From nas@python.ca Tue Apr 2 15:10:01 2002 From: nas@python.ca (Neil Schemenauer) Date: Tue, 2 Apr 2002 07:10:01 -0800 Subject: [Python-Dev] Re: PEP 279 In-Reply-To: ; from ping@lfw.org on Tue, Apr 02, 2002 at 05:25:45AM -0600 References: <000f01c1da0e$82fe8480$4c66accf@othello> Message-ID: <20020402071001.A28846@glacier.arctrix.com> Ka-Ping Yee wrote: > I prefer "enumerate" to "indexer" as well, though it's a bit long. > > "enum" would be nice. I don't like indexer either. It's a noun. I think I like enum best. enumerate is my second choice. Neil From nas@python.ca Tue Apr 2 15:13:55 2002 From: nas@python.ca (Neil Schemenauer) Date: Tue, 2 Apr 2002 07:13:55 -0800 Subject: [Python-Dev] Deprecating C APIs (Use of PyArg_NoArgs()) In-Reply-To: <3CA9C1A6.CB16C1ED@metaslash.com>; from neal@metaslash.com on Tue, Apr 02, 2002 at 09:35:18AM -0500 References: <3C9F4AC0.C076679B@metaslash.com> <3C9F8B0F.D921E530@metaslash.com> <200203252052.g2PKqJa08509@pcp742651pcs.reston01.va.comcast.net> <3C9FA9D1.DA46BE04@metaslash.com> <200203252300.g2PN0DY09544@pcp742651pcs.reston01.va.comcast.net> <3CA9777F.BB3DA0DD@lemburg.com> <3CA9C1A6.CB16C1ED@metaslash.com> Message-ID: <20020402071355.B28846@glacier.arctrix.com> Neal Norwitz wrote: > Perhaps, we could wrap all deprecated functions/macros in: > > #ifdef Py_DEPRECATED > #endif I like this idea. We could easily undefine it to ensure that the core code is not using deprecated APIs. Neil From trentm@ActiveState.com Tue Apr 2 16:54:47 2002 From: trentm@ActiveState.com (Trent Mick) Date: Tue, 2 Apr 2002 08:54:47 -0800 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Misc NEWS,1.146.2.9,1.146.2.10 In-Reply-To: <200204021427.g32ERNY06198@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Tue, Apr 02, 2002 at 09:27:23AM -0500 References: <200204020606.g3266Kd01556@mbuna.arbhome.com.au> <200204021427.g32ERNY06198@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020402085447.D10581@ActiveState.com> [Guido van Rossum wrote] > There's one issue: I'd rather not see Tim waste time on recreating the > Windows setup he used for the 2.1.2 installer. Maybe this can be a > source-only release? That's good enough for Zope (while Zope > technically supports Windows, nobody uses it this way, certainly not > the people who are likely to be bit by the above problem). Dunno if it makes any difference but it would be really easy for there to be an ActivePython 2.1.3 build for Windows. Any users desperate for a Windows installer (admittedly probably a small number) could be pointed to it. Trent -- Trent Mick TrentM@ActiveState.com From martin@v.loewis.de Tue Apr 2 17:15:48 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 02 Apr 2002 19:15:48 +0200 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Misc NEWS,1.146.2.9,1.146.2.10 In-Reply-To: <20020402085447.D10581@ActiveState.com> References: <200204020606.g3266Kd01556@mbuna.arbhome.com.au> <200204021427.g32ERNY06198@pcp742651pcs.reston01.va.comcast.net> <20020402085447.D10581@ActiveState.com> Message-ID: Trent Mick writes: > Dunno if it makes any difference but it would be really easy for there to be > an ActivePython 2.1.3 build for Windows. Any users desperate for a Windows > installer (admittedly probably a small number) could be pointed to it. I definitely think this would make a difference, so I'd encourage you to do so. Regards, Martin From mal@lemburg.com Tue Apr 2 17:19:01 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 02 Apr 2002 19:19:01 +0200 Subject: [Python-Dev] Freeze and shared modules Message-ID: <3CA9E805.AE012428@lemburg.com> Does anyone know which compiler/linker options have to be specified to make frozen Python interpreters correctly import shared modules ? Many of the mxCGIPython are running into this problem (the default settings in Python or freeze seem to be wrong on this account) and it got even worse with Python 2.2 where even the standard modules are built shared rather than statically linked. Thanks, -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From pobrien@orbtech.com Tue Apr 2 16:45:42 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Tue, 2 Apr 2002 10:45:42 -0600 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: <200204021340.g32Dekrq025410@ratthing-b246.strakt.com> Message-ID: [Laura Creighton] [powerful prose snipped] > I want to keep python out of the True and False business. Python cares > about whether a value is Something or Nothing. This is beautiful, and > _better_ than what the other languages do. Wow. I'm almost speechless. You've presented a unique point of view with such vividness that I'm quite literally stunned. If I could afford it I would pay you to evaluate every one of my ideas before I made a decision. Thank you for such a powerful argument. I can't predict whether it will change the course of events, but it sure has opened my eyes to a different perspective. I think it would be foolish to not ponder this issue for a while before making any final decisions. --- Patrick K. O'Brien Orbtech From guido@python.org Tue Apr 2 17:22:19 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 02 Apr 2002 12:22:19 -0500 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Misc NEWS,1.146.2.9,1.146.2.10 In-Reply-To: Your message of "Tue, 02 Apr 2002 08:54:47 PST." <20020402085447.D10581@ActiveState.com> References: <200204020606.g3266Kd01556@mbuna.arbhome.com.au> <200204021427.g32ERNY06198@pcp742651pcs.reston01.va.comcast.net> <20020402085447.D10581@ActiveState.com> Message-ID: <200204021722.g32HMJl19040@odiug.zope.com> > Dunno if it makes any difference but it would be really easy for > there to be an ActivePython 2.1.3 build for Windows. Any users > desperate for a Windows installer (admittedly probably a small > number) could be pointed to it. Thanks! --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Tue Apr 2 17:35:06 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 02 Apr 2002 19:35:06 +0200 Subject: [Python-Dev] Re: PEP 279 References: Message-ID: <3CA9EBCA.CA740AAF@lemburg.com> Tim Peters wrote: > > [Raymond Hettinger] > > Okay, here's what we have so far: > > > > iterindexed()-- five syllables is a mouthfull > > index() -- nice verb but could be confused the .index() method > > indexed() -- widely liked however adjectives should be avoided > > count() -- direct and explicit but often used in other contexts > > itercount() -- direct, explicit and hated by more than one person > > enumerate() -- a contender but doesn't mention iteration or indices > > iteritems() -- already used by dictionaries for key:value pairs > > Since Guido is sticking to bool() for his current PEP, I guess that leaves > truth() open for this . > > iotazip() is a natural for those with both APL and Haskell backgrounds, and > sounds way cool if you're burned out. > > ordinalize() is an even clumsier way to say enumerate(). > > poll() would confuse everyone almost equally. > > countoff() brings to mind a common isomorphic procedure in American schools > ("One!", "Two!", "Three!", ...). I kinda like it, although > zerobasedcountoff() may be more appropriate here . > > rollcall() is cute. > > paginate() is descriptive for one specific application. > > If indexed() is disliked just because it's an adjective, Google finds only > one hit for indicify(). indexify() gets 103. > > I'm amazed that Marc-Andre's irange() isn't already in the list (it's part > of his mxTools Python extension). As long as you remain backwards compatible with irange() I don't have objections ;-) (same for the other extra builtins in mxTools) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From martin@v.loewis.de Tue Apr 2 17:43:36 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 02 Apr 2002 19:43:36 +0200 Subject: [Python-Dev] Freeze and shared modules In-Reply-To: <3CA9E805.AE012428@lemburg.com> References: <3CA9E805.AE012428@lemburg.com> Message-ID: "M.-A. Lemburg" writes: > Does anyone know which compiler/linker options have to be specified > to make frozen Python interpreters correctly import shared modules ? LINKFORSHARED should be the right set of options. > Many of the mxCGIPython are running into this problem (the default > settings in Python or freeze seem to be wrong on this account) and > it got even worse with Python 2.2 where even the standard modules > are built shared rather than statically linked. Yes, building all extension modules as shared libraries is evil for freezing applications. I wish Linux distributors would recognize this and write proper Setup files, instead of relying on the default mechanism. Regards, Martin From mal@lemburg.com Tue Apr 2 18:49:41 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 02 Apr 2002 20:49:41 +0200 Subject: [Python-Dev] Re: PEP 285: Adding a bool type References: <2zRcLUAVjXp8EwVv@jessikat.fsnet.co.uk> <200204021340.g32Dekrq025410@ratthing-b246.strakt.com> Message-ID: <3CA9FD45.E6E33129@lemburg.com> I haven't followed this thread for quite some time, but since the PEP still seems alive, let me add some experience I've had with using the already existing singletons (Py_True and Py_False) in Python for recognizing truth values. Py_True and Py_False can be exposed in Python via True = (1==1) False = (1!=1) and most comparison operations and quite a few other APIs returning truth values make use of these singletons. I thought it would be a good idea to use those two singletons for protocols which use booleans such as XML-RPC. My experiences with this approach are, well, not so good :-/ The reason is that True and False are really integers and not of a special new type. Now they are singletons, which is good, since they represent the only states a boolean can have and in many cases work reasonably well as boolean representative, but not always (e.g. arithmetic operations such as True - True == False). Also, when trying to recognize the two singletons you have to use the "is" comparison -- "==" will fail to differentiate between 1 and True... def isboolean(x): return x in (True, False) ...doesn't work... def isboolean(x): return (x is True) or (x is False) ..does. As a conclusion, I think it would be better to make bool() a new type which does *not* inherit from integers, but which does know how deal with other types which are commonly used together with booleans such as integers. However, the type should implement boolean algebra and not try to mimic integer arithemtic, i.e. True - True raises an exception. Py_True and Py_False should then be made singletons of this new type (possibly after a transition phase which replaces the singletons with a version that doesn't raise exceptions but instead issues warnings). This may sound painful at first, but in the long run, I believe, it'll result in the same benefits as other painful changes have or will (e.g. the change from integer division to floating point division). -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From mal@lemburg.com Tue Apr 2 19:02:51 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 02 Apr 2002 21:02:51 +0200 Subject: [Python-Dev] Deprecating C APIs (Use of PyArg_NoArgs()) References: <3C9F4AC0.C076679B@metaslash.com> <3C9F8B0F.D921E530@metaslash.com> <200203252052.g2PKqJa08509@pcp742651pcs.reston01.va.comcast.net> <3C9FA9D1.DA46BE04@metaslash.com> <200203252300.g2PN0DY09544@pcp742651pcs.reston01.va.comcast.net> <3CA9777F.BB3DA0DD@lemburg.com> <3CA9C1A6.CB16C1ED@metaslash.com> <20020402071355.B28846@glacier.arctrix.com> Message-ID: <3CAA005B.24C9243F@lemburg.com> Neil Schemenauer wrote: > > Neal Norwitz wrote: > > Perhaps, we could wrap all deprecated functions/macros in: > > > > #ifdef Py_DEPRECATED > > #endif > > I like this idea. We could easily undefine it to ensure that the core > code is not using deprecated APIs. FYI, I have been using a mxpyapi.h header file for this the last couple of years (see e.g. mxDateTime). Perhaps Python should ship with one of these as well, e.g. pycompat.h. This include file would only be used by extensions wishing to support older Python versions, of course, not by the Python core itself. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From mal@lemburg.com Tue Apr 2 19:12:16 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 02 Apr 2002 21:12:16 +0200 Subject: [Python-Dev] Deprecating C APIs (Use of PyArg_NoArgs()) References: <3C9F4AC0.C076679B@metaslash.com> <3C9F8B0F.D921E530@metaslash.com> <200203252052.g2PKqJa08509@pcp742651pcs.reston01.va.comcast.net> <3C9FA9D1.DA46BE04@metaslash.com> <200203252300.g2PN0DY09544@pcp742651pcs.reston01.va.comcast.net> <3CA9777F.BB3DA0DD@lemburg.com> <200204021437.g32Eb7C09097@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CAA0290.24D1E302@lemburg.com> Guido van Rossum wrote: > > > +1 on removing PyArg_GetInt() > > OK. > > > -1 on removing PyArg_NoArgs() ... this API is still in very active > > use in third party extensions. > > Would it be OK if it issued a warning? Hmm, people would start screaming, I guess -- this API is used far more than e.g. the regex/regsub module combo. A one-time-per- module warning would probably be OK though. I seriously think that we should install a set of common third party extensions somewhere to that we can easily check usage of the APIs in the outside world, e.g. take the packages Aahz collected recently and unzip them to some directory on python.org. A grep would then quickly produce proof of "common" usage. > > I think we should come up with a better way to deprecate C level > > APIs. NEWS is not as effective as one might wish; any deprecation > > or removal of an API should also include upgrade information > > for the C extension writer so that upgrading their modules > > does not require diving into the new style C API code every > > time. > > In this case, PyArg_ParseTuple(args, ":functionName") is pretty simple > to explain. > > > I'd suggest to have a PEP which documents all C API changes > > or deprecations together with a short decsription of how to > > update old code. > > It shouldn't be a PEP (that's a *proposal*). It should be part of the > C API docs, possibly an appendix. I was thinking of an informational PEP -- mostly because PEPs are easily accessible and also simple to write (LaTeX is not everybody's friend ;-). > > Another aspect which we ought to consider is that C extension > > writers may have a different view of when to leave backward > > compatibility path in favour of a new Python version. IMHO, > > there should always be a way to emulate old behaviour so that > > extensions can continue to support Python 1.5.2 if they need to. > > Not a problem in this case. True, but there are other instances where this was a problem in the past, e.g. #if PY_VERSION_HEX < 0x01060000 #if !defined(PyObject_DEL) # define PyObject_DEL(x) free(x) # define PyObject_Del(x) free(x) #endif #endif -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From pedroni@inf.ethz.ch Tue Apr 2 19:12:08 2002 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Tue, 2 Apr 2002 21:12:08 +0200 Subject: [Python-Dev] Re: PEP 285: Adding a bool type References: <2zRcLUAVjXp8EwVv@jessikat.fsnet.co.uk> <200204021340.g32Dekrq025410@ratthing-b246.strakt.com> <3CA9FD45.E6E33129@lemburg.com> Message-ID: <001101c1da7a$605da660$6d94fea9@newmexico> From: M.-A. Lemburg > As a conclusion, I think it would be better to make bool() a > new type which does *not* inherit from integers, but which > does know how deal with other types which are commonly > used together with booleans such as integers. However, the > type should implement boolean algebra and not try to > mimic integer arithemtic, i.e. True - True raises an > exception. > "but which does know how deal with other types which are commonly used together with booleans such as integers" ?? with deal you mean throwing exceptions? regards. From DavidA@ActiveState.com Tue Apr 2 19:43:32 2002 From: DavidA@ActiveState.com (David Ascher) Date: Tue, 02 Apr 2002 11:43:32 -0800 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Misc NEWS,1.146.2.9,1.146.2.10 References: <200204020606.g3266Kd01556@mbuna.arbhome.com.au> <200204021427.g32ERNY06198@pcp742651pcs.reston01.va.comcast.net> <20020402085447.D10581@ActiveState.com> Message-ID: <3CAA09E4.1F89248F@activestate.com> > > Dunno if it makes any difference but it would be really easy for there to be > > an ActivePython 2.1.3 build for Windows. Any users desperate for a Windows > > installer (admittedly probably a small number) could be pointed to it. > > I definitely think this would make a difference, so I'd encourage you > to do so. Let me follow-up. If there is a 2.1.3 source release, there will definitely be a 2.1.3 ActivePython release. --da From aahz@pythoncraft.com Tue Apr 2 19:46:26 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 2 Apr 2002 14:46:26 -0500 Subject: [Python-Dev] Deprecating C APIs (Use of PyArg_NoArgs()) In-Reply-To: <3CAA0290.24D1E302@lemburg.com> References: <3C9F4AC0.C076679B@metaslash.com> <3C9F8B0F.D921E530@metaslash.com> <200203252052.g2PKqJa08509@pcp742651pcs.reston01.va.comcast.net> <3C9FA9D1.DA46BE04@metaslash.com> <200203252300.g2PN0DY09544@pcp742651pcs.reston01.va.comcast.net> <3CA9777F.BB3DA0DD@lemburg.com> <200204021437.g32Eb7C09097@pcp742651pcs.reston01.va.comcast.net> <3CAA0290.24D1E302@lemburg.com> Message-ID: <20020402194625.GA28632@panix.com> On Tue, Apr 02, 2002, M.-A. Lemburg wrote: > > I seriously think that we should install a set of common third party > extensions somewhere to that we can easily check usage of the APIs in > the outside world, e.g. take the packages Aahz collected recently and > unzip them to some directory on python.org. A grep would then quickly > produce proof of "common" usage. In case anyone wants to take up MAL's idea, here's an alphabetical list of all the extensions I recognized in that thread: 4Suite calldll Gadfly HTMLgen mxODBC NumPy piddle PIL PyGame PyOpenGL PythonCard SWIG win32all wxPython ZODB -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ Why is this newsgroup different from all other newsgroups? From guido@python.org Tue Apr 2 20:39:31 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 02 Apr 2002 15:39:31 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format In-Reply-To: Your message of "Tue, 02 Apr 2002 00:28:17 EST." References: Message-ID: <200204022039.g32KdWZ19941@odiug.zope.com> > Here's a serious proposal, safe to post now that April Fool's is > over. Please read & comment. Good PEP, David! What's the next step? Should the processing code be incorporated in the standard library? Should we start converting the standard library docs to reStructuredText? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 2 21:04:51 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 02 Apr 2002 16:04:51 -0500 Subject: [Python-Dev] Deprecating C APIs (Use of PyArg_NoArgs()) In-Reply-To: Your message of "Tue, 02 Apr 2002 21:12:16 +0200." <3CAA0290.24D1E302@lemburg.com> References: <3C9F4AC0.C076679B@metaslash.com> <3C9F8B0F.D921E530@metaslash.com> <200203252052.g2PKqJa08509@pcp742651pcs.reston01.va.comcast.net> <3C9FA9D1.DA46BE04@metaslash.com> <200203252300.g2PN0DY09544@pcp742651pcs.reston01.va.comcast.net> <3CA9777F.BB3DA0DD@lemburg.com> <200204021437.g32Eb7C09097@pcp742651pcs.reston01.va.comcast.net> <3CAA0290.24D1E302@lemburg.com> Message-ID: <200204022104.g32L4pC20305@odiug.zope.com> > > > -1 on removing PyArg_NoArgs() ... this API is still in very active > > > use in third party extensions. > > > > Would it be OK if it issued a warning? > > Hmm, people would start screaming, I guess -- this API is used > far more than e.g. the regex/regsub module combo. A one-time-per- > module warning would probably be OK though. That's what I wanted, but now that you mention it I'm not sure if the warning module can currrently detect from which module a particular C API is called -- it'll probably put the blame on a particular line in the user's code, and then repeat the warning for each *different* line of user code. > I seriously think that we should install a set of common > third party extensions somewhere to that we can easily > check usage of the APIs in the outside world, e.g. take > the packages Aahz collected recently and unzip them to > some directory on python.org. A grep would then quickly > produce proof of "common" usage. +1 > > > I'd suggest to have a PEP which documents all C API changes > > > or deprecations together with a short decsription of how to > > > update old code. > > > > It shouldn't be a PEP (that's a *proposal*). It should be part of > > the C API docs, possibly an appendix. > > I was thinking of an informational PEP -- mostly because PEPs > are easily accessible and also simple to write (LaTeX is not > everybody's friend ;-). Yeah, that's fine. Let whoever writes it decide on the format. :-) > True, but there are other instances where this was a problem > in the past, e.g. > > #if PY_VERSION_HEX < 0x01060000 > > #if !defined(PyObject_DEL) > # define PyObject_DEL(x) free(x) > # define PyObject_Del(x) free(x) > #endif > > #endif Why the PY_VERSION_HEX guard? Isn't the !defined() guard enough? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 2 21:45:10 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 02 Apr 2002 16:45:10 -0500 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: Your message of "Tue, 02 Apr 2002 20:49:41 +0200." <3CA9FD45.E6E33129@lemburg.com> References: <2zRcLUAVjXp8EwVv@jessikat.fsnet.co.uk> <200204021340.g32Dekrq025410@ratthing-b246.strakt.com> <3CA9FD45.E6E33129@lemburg.com> Message-ID: <200204022145.g32LjAK20525@odiug.zope.com> > I haven't followed this thread for quite some time, but since the > PEP still seems alive, :-) > let me add some experience I've had with > using the already existing singletons (Py_True and Py_False) > in Python for recognizing truth values. > > Py_True and Py_False can be exposed in Python via > > True = (1==1) > False = (1!=1) Yuck. What a horrible implementation detail. > and most comparison operations and quite a few other APIs > returning truth values make use of these singletons. This works purely by accident. Given that at least Py_False is also known as &_Py_ZeroStruct, I'm surprised that the code that optimizes ints doesn't initialize small_ints[NSMALLNEGINTS] with Py_False. > I thought it would be a good idea to use those two > singletons for protocols which use booleans such as > XML-RPC. My experiences with this approach are, well, > not so good :-/ > > The reason is that True and False are really integers > and not of a special new type. Now they are singletons, > which is good, since they represent the only states > a boolean can have and in many cases work reasonably > well as boolean representative, but not always > (e.g. arithmetic operations such as True - True == False). > Also, when trying to recognize the two singletons you have > to use the "is" comparison -- "==" will fail to > differentiate between 1 and True... > > def isboolean(x): > return x in (True, False) > > ...doesn't work... > > def isboolean(x): > return (x is True) or (x is False) > > ..does. The correct way if PEP 285 is accepted would of course be isinstance(x, bool). > As a conclusion, I think it would be better to make bool() a > new type which does *not* inherit from integers, but which > does know how deal with other types which are commonly > used together with booleans such as integers. However, the > type should implement boolean algebra and not try to > mimic integer arithemtic, i.e. True - True raises an > exception. That would break more code. E.g. (a!=0) + (b!=0) + (c!=0) counts how many of (a, b, c) are nonzero; this would break. > Py_True and Py_False should then be made singletons > of this new type (possibly after a transition phase which > replaces the singletons with a version that doesn't raise > exceptions but instead issues warnings). That's what the PEP does (but with a different type). > This may sound painful at first, but in the long run, > I believe, it'll result in the same benefits as other > painful changes have or will (e.g. the change from integer > division to floating point division). I don't see the introduction of the bool type as painful at all. I find getting it accepted painful though. :-( --Guido van Rossum (home page: http://www.python.org/~guido/) From martin@v.loewis.de Tue Apr 2 22:38:44 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 03 Apr 2002 00:38:44 +0200 Subject: [Python-Dev] Deprecating C APIs (Use of PyArg_NoArgs()) In-Reply-To: <200204022104.g32L4pC20305@odiug.zope.com> References: <3C9F4AC0.C076679B@metaslash.com> <3C9F8B0F.D921E530@metaslash.com> <200203252052.g2PKqJa08509@pcp742651pcs.reston01.va.comcast.net> <3C9FA9D1.DA46BE04@metaslash.com> <200203252300.g2PN0DY09544@pcp742651pcs.reston01.va.comcast.net> <3CA9777F.BB3DA0DD@lemburg.com> <200204021437.g32Eb7C09097@pcp742651pcs.reston01.va.comcast.net> <3CAA0290.24D1E302@lemburg.com> <200204022104.g32L4pC20305@odiug.zope.com> Message-ID: Guido van Rossum writes: > That's what I wanted, but now that you mention it I'm not sure if the > warning module can currrently detect from which module a particular C > API is called -- it'll probably put the blame on a particular line in > the user's code, and then repeat the warning for each *different* line > of user code. What *could* be done, of course, is to give the warning at most once. Regards, Martin From klm@zope.com Tue Apr 2 23:47:13 2002 From: klm@zope.com (Ken Manheimer) Date: Tue, 2 Apr 2002 18:47:13 -0500 (EST) Subject: [Python-Dev] Re: PEP 279 In-Reply-To: Message-ID: On Tue, 2 Apr 2002, Tim Peters wrote: > > So now I'd like to choose between enumerate() and indexer(). Any > > closing arguments? > > I'm leaving that quote anonymous so nobody gives it more or less weight than > it deserves . I prefer enumerate(), because a transitive verb is more > appropriate than a noun. OTOH, enumerate() is just a fancy pants way of > saying "countoff()", which is nicely confusable with operator.countOf if > you're tired . I know i'm late for this party, but i'd like to introduce 'itemize()'. "Everybody, itemize(). Itemize(), everybody. Pleased to meetcha, i'm sure." Nice thing about itemize is it turns values into key/value tuples, much like .items() on various sequences. -- Ken klm@zope.com From klm@zope.com Tue Apr 2 23:50:25 2002 From: klm@zope.com (Ken Manheimer) Date: Tue, 2 Apr 2002 18:50:25 -0500 (EST) Subject: [Python-Dev] Re: PEP 279 In-Reply-To: Message-ID: On Tue, 2 Apr 2002, Ken Manheimer wrote: > On Tue, 2 Apr 2002, Tim Peters wrote: > > > > So now I'd like to choose between enumerate() and indexer(). Any > > > closing arguments? > > > > I'm leaving that quote anonymous so nobody gives it more or less weight than > > it deserves . I prefer enumerate(), because a transitive verb is more > > appropriate than a noun. OTOH, enumerate() is just a fancy pants way of > > saying "countoff()", which is nicely confusable with operator.countOf if > > you're tired . > > I know i'm late for this party, but i'd like to introduce 'itemize()'. > > "Everybody, itemize(). Itemize(), everybody. Pleased to meetcha, i'm > sure." > > Nice thing about itemize is it turns values into key/value tuples, much > like .items() on various sequences. Whoops, twice - ".items() on mappings", and i was wrong, besides, as raymond pointed out... -- Ken klm@zope.com From goodger@users.sourceforge.net Wed Apr 3 00:43:25 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Tue, 02 Apr 2002 19:43:25 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format In-Reply-To: <200204022039.g32KdWZ19941@odiug.zope.com> Message-ID: Guido van Rossum wrote: Guido van Rossum wrote: > Good PEP, David! Thanks! I'll take the generally positive tone of your reply as a good sign. > What's the next step? Mark the PEP as "Accepted"? But first, I'd like to address any issues that may be raised. I'd like to give people a few days at least to respond. Does a lack of responses mean that there *are* no issues? I came here for a good argument! Seriously though, just accepting the PEP would be great progress. > Should the processing code be incorporated in the standard library? It's not ready for that yet. Here's a summary of the state of the code: - The project is split in two at present: the parser and everything else (the DPS proper). They are to be merged & renamed to "Docutils". This will remove some artificial complexity, reduce some redundancy especially with the test code, and make the whole thing much easier to install. - The parser is functionally complete for standalone documents, and works quite well (passes all 300 unit tests). However, - The code needs some serious refactoring in places. - The internal documentation needs to be completed. - The non-parser part (current DPS) is still in its infancy. It's currently only able to process standalone documents into simple HTML. - The docstring extraction & processing part of Docutils (what I call the "Python source reader" component) is nowhere near ready. Implementing the Python roles for "interpreted text" (links based on namespace context) will need a significant effort. But that has more to do with PEP 258. BTW, expect PEP revisions soon, Barry! - There's no support for PEP processing yet. Assuming that this part of the PEP is accepted, - The PEP strategy for section headers must first be decided (as-is plus reStructuredText in the indented body text, or replace with underline syntax and drop the indents, or allow both). - The parser needs support for PEP-specific constructs (RFC 2822 headers; recognize "PEP \d+" and "RFC[- ]?\d+" as links; Q&A). - pep2html.py will need some work. It would become a front-end to a "PEP reader" component. I think the Docutils code should continue to be developed separately from the stdlib, until it is worthy. It may or may not be ready in time for 2.3, depending on the usual factors: my time, getting more interested developers on board, etc. The PEP processing part could be installed before the full docstring processing part is ready; that should be doable for 2.3. > Should we start converting the standard library docs to > reStructuredText? I assume you mean "docstrings"? Oherwise Fred may take exception, at least until LaTeX support is in place. ;-) There's no rush to convert library docstrings. There won't be a benefit for a while, but there's no danger either. -- David Goodger goodger@users.sourceforge.net Open-source projects: - Python Docstring Processing System: http://docstring.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net - The Go Tools Project: http://gotools.sourceforge.net From goodger@users.sourceforge.net Wed Apr 3 02:27:05 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Tue, 02 Apr 2002 21:27:05 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format In-Reply-To: Message-ID: I spoke too soon: > Does a lack of responses mean that there *are* no issues? > > I came here for a good argument! And I got one, on comp.lang.python, flamebait and all. BTW, Barry Warsaw wrote this comment for CVS revision 1.7 of python/nondist/peps/pep-0009.txt: "In David Goodger's PEP 287, he has a better Emacs turd section." I hope a new revision gets checked in soon. I don't want to be Google-associated with "a better Emacs turd". Drat, that's done it. Now my fate is sealed. -- David Goodger goodger@users.sourceforge.net Open-source projects: - Python Docstring Processing System: http://docstring.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net - The Go Tools Project: http://gotools.sourceforge.net From Anthony Baxter Wed Apr 3 03:23:57 2002 From: Anthony Baxter (Anthony Baxter) Date: Wed, 03 Apr 2002 13:23:57 +1000 Subject: [Python-Dev] 2.1.3 call for patches. Message-ID: <200204030323.g333NvL15986@mbuna.arbhome.com.au> I plan to cut a 2.1.3 release early next week (Monday morning australian eastern time). Here's your chance to nominate bug fixes that should go into the baseline code... (I don't plan to be hunting through changelogs again...) Soooo, speak or miss out. Anthony From tim.one@comcast.net Wed Apr 3 03:34:17 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 02 Apr 2002 22:34:17 -0500 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Misc NEWS,1.146.2.9,1.146.2.10 In-Reply-To: <200204021427.g32ERNY06198@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido, on a 2.1.3 release] > ... > AFAIC you can release exactly what's in the release21-maint > branch now -- only update the README and LICENSE file. A reminder that if that's all there is to it, it's more efficient for one of the PLabs guys to do it (at least one of us will be involved anyway, and *coordinating* such a little bit of work would drag on longer than just doing it). From fdrake@acm.org Wed Apr 3 03:50:07 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 2 Apr 2002 22:50:07 -0500 Subject: [Python-Dev] New canonical location for PEPs Message-ID: <15530.31727.557290.92540@grendel.zope.com> Anyone closely following today's checkinsmay have noticed that the online PEPs are now hosted on python.org rather than python.sf.net. The new "canonical" location is: http://www.python.org/peps/ Links that used the old re-director on python.org will still work, and the HTML versions of the PEPs on python.sf.net have been replaced with relocation notices that redirect after 10 seconds. There are no changes to the way PEPs will generally be managed, but this does change how and (more significantly) *who* can actually get new & updated PEPs to appear online. People with commit access to the python.org CVS will be able to push new versions to the site, and this still needs to be done manually. I'll get a cron job installed which updates from CVS, after which point it will just be a matter of waiting for the cron job to run. I expect it to run frequently enough that we won't spend a *lot* of time waiting for new PEPs to appear online. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From fdrake@acm.org Wed Apr 3 04:39:01 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 2 Apr 2002 23:39:01 -0500 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Misc NEWS,1.146.2.9,1.146.2.10 In-Reply-To: References: <200204021427.g32ERNY06198@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15530.34661.788683.616942@grendel.zope.com> Tim Peters writes: > A reminder that if that's all there is to it, it's more efficient for one of > the PLabs guys to do it (at least one of us will be involved anyway, and For the docs, I'll plan on doing an HTML-only release, similar to the recent 2.2p1 package. The only real reason to do that is that the PEP links changed, but there's only one in that version of the documentation. It makes sense to update the online material, but it makes no difference for the typeset versions. The other changes are pretty minor, at best. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From greg@cosc.canterbury.ac.nz Wed Apr 3 05:14:29 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Wed, 03 Apr 2002 17:14:29 +1200 (NZST) Subject: [Python-Dev] Re: PEP 279 In-Reply-To: <200204021357.g32DvTj06129@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204030514.RAA00472@s454.cosc.canterbury.ac.nz> Guido: > 0. itemize > 1. enumerate I'd like to add: 2. number It says pretty much what we mean -- number these items, i.e. assign a number to each one. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From jafo-python-dev@tummy.com Wed Apr 3 05:18:39 2002 From: jafo-python-dev@tummy.com (Sean Reifschneider) Date: Tue, 2 Apr 2002 22:18:39 -0700 Subject: [Python-Dev] 2.2.1c2 RPMs Message-ID: <20020402221838.G16962@tummy.com> I've uploaded the RPMs for 2.2.1rc2. This is what the final 2.2.1 beta will be based on, so if you have any final comments let me know. You can snag them at: ftp://ftp.tummy.com/pub/tummy/RPMS/SRPMS/python2-2.2.1c2-1.src.rpm ftp://ftp.tummy.com/pub/tummy/RPMS/binaries-KRUD-7.2-i386/python2-2.2.1c2-1.i386.rpm ftp://ftp.tummy.com/pub/tummy/RPMS/binaries-KRUD-7.2-i386/python2-devel-2.2.1c2-1.i386.rpm ftp://ftp.tummy.com/pub/tummy/RPMS/binaries-KRUD-7.2-i386/python2-tkinter-2.2.1c2-1.i386.rpm ftp://ftp.tummy.com/pub/tummy/RPMS/binaries-KRUD-7.2-i386/python2-tools-2.2.1c2-1.i386.rpm ftp://ftp.tummy.com/pub/tummy/RPMS/binaries-KRUD-7.2-i386/python2-docs-2.2.1c2-1.i386.rpm Thanks, Sean -- Read error: 666 (Connection reset by Satan) Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From martin@v.loewis.de Wed Apr 3 06:29:20 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 03 Apr 2002 08:29:20 +0200 Subject: [Python-Dev] New canonical location for PEPs In-Reply-To: <15530.31727.557290.92540@grendel.zope.com> References: <15530.31727.557290.92540@grendel.zope.com> Message-ID: "Fred L. Drake, Jr." writes: > I'll get a cron job installed which updates from CVS, after which > point it will just be a matter of waiting for the cron job to run. I > expect it to run frequently enough that we won't spend a *lot* of time > waiting for new PEPs to appear online. I wonder whether some 'bot could watch the checkin messages, and update the PEPs online if it sees a checkin message for the PEPs. Regards, Martin From jafo-python-dev@tummy.com Wed Apr 3 06:37:22 2002 From: jafo-python-dev@tummy.com (Sean Reifschneider) Date: Tue, 2 Apr 2002 23:37:22 -0700 Subject: [Python-Dev] New canonical location for PEPs In-Reply-To: <15530.31727.557290.92540@grendel.zope.com>; from fdrake@acm.org on Tue, Apr 02, 2002 at 10:50:07PM -0500 References: <15530.31727.557290.92540@grendel.zope.com> Message-ID: <20020402233722.H16962@tummy.com> On Tue, Apr 02, 2002 at 10:50:07PM -0500, Fred L. Drake, Jr. wrote: >expect it to run frequently enough that we won't spend a *lot* of time >waiting for new PEPs to appear online. There is apparently a way that you can make CVS fire off some action when a commit is done... It looks like it might be possible with the "CVS/Checkin.prog". From the CVS man page: CVS/Checkin.prog Name of program to run on `cvs commit'. Sean -- -rw-rw-rw-: File permissions of the beast. (Chuck Phillips) mknod /dev/beast c 6 66 Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From martin@v.loewis.de Wed Apr 3 07:28:17 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 03 Apr 2002 09:28:17 +0200 Subject: [Python-Dev] New canonical location for PEPs In-Reply-To: <20020402233722.H16962@tummy.com> References: <15530.31727.557290.92540@grendel.zope.com> <20020402233722.H16962@tummy.com> Message-ID: Sean Reifschneider writes: > There is apparently a way that you can make CVS fire off some action when a > commit is done... It looks like it might be possible with the > "CVS/Checkin.prog". From the CVS man page: > > CVS/Checkin.prog > Name of program to run on `cvs commit'. This is already used to send the commit message email. However, it is really hard to get this mechanism to update web pages on a different machine (especially under a different account than the committer's account). Leaving security aside, the commit program runs with the CVS lock still being hold by the committer. If the commit program tries to update a CVS tree on a different machine, you get a deadlock. Regards Martin From jafo-python-dev@tummy.com Wed Apr 3 07:49:49 2002 From: jafo-python-dev@tummy.com (Sean Reifschneider) Date: Wed, 3 Apr 2002 00:49:49 -0700 Subject: [Python-Dev] New canonical location for PEPs In-Reply-To: ; from martin@v.loewis.de on Wed, Apr 03, 2002 at 09:28:17AM +0200 References: <15530.31727.557290.92540@grendel.zope.com> <20020402233722.H16962@tummy.com> Message-ID: <20020403004949.I16962@tummy.com> On Wed, Apr 03, 2002 at 09:28:17AM +0200, Martin v. Loewis wrote: >really hard to get this mechanism to update web pages on a different >machine (especially under a different account than the committer's That's what ssh with public/private key pairs and "command=" in authorized_keys is for. :-) There is some reasonable mechanism for doing this, as it *IS* done in some places. The guy who had set it up for one web site and was therefore supposed to be helping get it set up on a local LUG web site has thus far not done so. :-( Sean -- Money is the root of all evil! Man needs roots... Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From tanzer@swing.co.at Wed Apr 3 07:01:01 2002 From: tanzer@swing.co.at (Christian Tanzer) Date: Wed, 03 Apr 2002 09:01:01 +0200 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: Your message of "Mon, 01 Apr 2002 16:21:39 CDT." <200204012121.g31LLdk00743@pcp742651pcs.reston01.va.comcast.net> Message-ID: > > Nice tease. But Guido's hammer with regard to future strictification > > of bool is exactly what makes me uneasy. > > Can you elaborate that? I thought I made it pretty clear I didn't > want to strictify bool (though I did open up the question as to > whether other people want that). Is that what you call a hammer? You made it clear enough. There are two things making me uneasy though: - A lot of people instantly started complaining about the new bools being not strict enough. Over time, all that clamour might change your POV. - I don't really understand why you proposed this PEP in the first place. For me, Python's (old) way of handling boolean conditions is one of its great strengths. This wasn't always the case. When I started using Python, I was slightly put off by the lack of a boolean data type. But quite quickly I learned to appreciate Python's way of dealing with this issue. All of that was implicit knowledge, though. It took Laura Creighton's well written post to make it explicit for me (thank you, Laura). After reading Alex Martelli's and Laura's arguments, I'm convinced that Python would be better off without this PEP and change to -1. Glad-that-I'm-not-a-language-designer-ly yr's, Christian -- = Christian Tanzer tanzer@swing.co.= at Glasauergasse 32 Tel: +43 1 876 62 = 36 A-1130 Vienna, Austria Fax: +43 1 877 66 = 92 From tim.one@comcast.net Wed Apr 3 08:20:04 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 03 Apr 2002 03:20:04 -0500 Subject: [Python-Dev] Suggested memory API rules for 2.3 Message-ID: Guido and I sat down and hashed this out Tuesday afternoon. Here's what I think Revealed Truth was. Indented text is intended to show up in the docs (after Fred reworks for sanity). Unindented text (like this) is commentary. Four technical words the docs really need are used freely: shall - mandatory; all bets are off if you don't shall not - forbidden; all bets are off if you do should - recommended; may become mandatory later should not - discouraged; may become forbidden later Nothing is deprecated for 2.3. The layers of PyCore_XYZ macros Neil already nuked are plain gone, and weren't documented anyway. Nothing has been added to the public API (wrt 2.2 as a base). These are the functions in the "raw memory" and "object memory" APIs: The raw memory API: A. PyMem_{Malloc, Realloc, Free} B. PyMem_{MALLOC, REALLOC, FREE} C. PyMem_{New, Resize, Del} D. PyMem_{NEW, RESIZE, DEL} The object memory API: E. PyObject_{Malloc, Realloc, Free} F. PyObject_{MALLOC, REALLOC, FREE} G. PyObject_{New, NewVar, Del} H. PyObject_{NEW, NEWVAR, DEL} Guido suggested leaving the PyObject_GC_xyz family out of this discussion, as it's clean already, self-contained, can't mix with *any* of the guys above, and has no compatibility nightmares to contend with (bless you, Neil). Function versus Macro Names Names ending with a mixed-case suffix are called "function names" (lines A, C, E, G). Names ending with an all-uppercase suffix are called "macro names" (lines B, D, F, H). This is for convenience in description, and does not necessarily reflect how a given name is implemented in C. Programs should use function names. Programs should not use macro names. Uses of function names are guaranteed binary compatible across releases; no such guarantee is given for uses of macro names. Yes, this means that almost all old code is now considered to be in bad style, and you're on official notice that the macro names *may* be deprecated down the road. I don't expect this to happen before 1.5.2 is no longer of significant interest. "Programs" does not include the Python core. We can do anything we need to do in the core, based on our complete knowledge of each release's internal implementation details. Programs currently using a macro name should switch to the corresponding function name (for example, replace PyMem_FREE with PyMem_Free). In 2.3 and beyond, it's guaranteed that corresponding function and macro names have the same input-output behavior. Before 2.3, PyMem_MALLOC and PyMem_REALLOC did not define what happened when passed a 0 argument. Programs should not assume that macro name spellings "are faster" than function name spellings. In most cases they aren't, and the cases in which they are may vary across releases. Macro names are retained only for backward compatibility. We need to change PyMem_MALLOC and PyMem_REALLOC for 2.3. It's going to be impossible for people to keep the rules straight if corresponding function and macro versions ever have different basic semantics. Since the macro names are not yet deprecated, they need to be fixed where they're currently broken in this way. Mixing and Matching After memory has been obtained via one of these functions, it should be resized and freed only by a function from the same line, except that PyMem_Free may be used freely in place of PyMem_Del, and PyObject_Free in place of PyObject_Del. Maintaining the Free/Del pseudo-distinction is pointless. For backward compatibility, memory obtained via the object memory family can be freed by any of Py{Mem, Object}_{Free, FREE, Del, DEL}. Mixing functions from the object family with the raw memory family is likely to become deprecated, Memory obtained by PyMem_{Malloc, MALLOC} shall be resized only by PyMem_{Realloc, REALLOC}. Memory obtained by PyMem_{New, NEW} shall be resized only by PyMem_{Resize, RESIZE}. Memory obtained by PyObject_{Malloc, MALLOC} shall be resized only by PyObject_{Realloc, REALLOC}. Note that the eight ways to spell "free" all have to map to the pymalloc free when pymalloc is enabled in 2.3. There is no way to spell "give me the raw platform free(), damn it", except for "free". If we think it's important to have a such a way in the core, it should be added to the private API. PyObject_{Malloc, MALLOC, Realloc, REALLOC} should make the same guarantees about 0 arguments as PyMem_xyz make. I'll have to dig into that, and, e.g., obmalloc.c's realloc currently goes out of its way *not* to treat 0 the way the Python docs promise PyMem_Realloc works (I had already added an XXX comment to the code there, but have not yet repaired it -- mostly because "the rules" were still up in the air). Relationship to Platform Allocator All names in lines A, B, C and D ultimately invoke the platform C malloc, realloc, or free. However, programs shall not mix any of these names with direct calls to the platform malloc, calloc, realloc, or free referencing the same base memory addresses, as Python may need to perform bookkeeping of its own around its calls to the platform allocator. We're not promising that it's going to be easy to play with some other implementation of malloc/realloc/free. We get one person per year who tries that, and they're usually trying a package that (sensibly enough) supplies its own macro replacements for the tokens "free", "malloc" and "realloc" specifically. For that reason, having our own macros like PyYAGNI_InRealLifeThisAlwaysExpandsToFree is pretty pointless; if you want to use some other malloc/realloc/free package, recompile everything using that package's header files. Note that all names in E, F, G and H invoke the Python object allocator. The Python object allocator may or may not invoke the platform allocator, and the object allocator may vary depending on configuration options. We don't promise anything about the internals of the object allocator. Threading Rules Programs shall hold the global interpreter lock before invoking a function in the object allocator (E, F, G, H). Programs need not, but may, hold the global interpreter lock before invoking a function in the raw memory allocator (A, B, C, D), except that a program shall hold the global interpreter lock before invoking PyMem_{Free, FREE, Del, DEL} on memory originally obtained from the object allocator (but note that a program should not mix calls this way). The conditions under which the GIL need not be held need also to be documented in the thread/GIL docs. Use as Function Designators Programs shall not use use a name from line B, C, D, F, G or H in a C context requiring a function designator. Names from lines A and E may be used in C contexts requiring a function designator. A function designator is a C expression having function type; C contexts requiring a function designator include as an actual argument to a function taking a function argument. This implies the A and E names have to expand to function names (not function applications, unless we're crazy). It wouldn't make sense to *require* this of "macro names". Many of the other "function names" *have* to be implemented as macros expanding to non-function-designator expressions, because they're specified to return a result of a type given by one of their arguments, and you just can't spell that in C without a macro to cast to the required type. Recommended Practice When backward compatibility is not a concern, using this subset of the raw memory and object memory APIs is recommended: PyMem_{Malloc, Realloc, Free} PyObject_{Malloc, Realloc, Free} PyObject_{New, NewVar} Always use PyMem functions on memory obtained from a PyMem function, and likewise for PyObject. Uses of PyMem_{New, Resize} are probably clearer if you do the multiplication and call the platform malloc/realloc yourself (although be aware that platform malloc/realloc vary in how they handle 0 arguments). Their existence in the Python C API dates back to a time when object allocation was handled in a different way. There's no reason not to use PyObject_Del, but it's the same as PyObject_Free, and using PyObject_Free instead should remind you that it does nothing except release the memory. From fredrik@pythonware.com Wed Apr 3 09:18:16 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 3 Apr 2002 11:18:16 +0200 Subject: [Python-Dev] PEP 287: reStructuredText Standard DocstringFormat References: Message-ID: <013901c1daf0$e04e2b30$ced241d5@hagrid> david wrote: > Does a lack of responses mean that there *are* no issues? the first time, I got to the following complete nonsense: "XML and SGML /.../ are verbose, difficult to type, and too cluttered to read comfortably as source." and having written several books in SGML and XML without noticing any of those "widely known" problems, I decided that it wasn't meaningful to continue. ::: after a second attempt to read it, I got stuck in the Q&A section. I've never seen such an arrogant PEP before; the authors clearly have very little experience from the problem domain (not only writing and maintaining documentation with markup, but also what makes Python so incredibly usable), yet they want to want to force their new invention down everyone's throat (hey, we spent a lot of time desiging this, so of course you shall have to use it): want to contribute a PEP? sorry, you have to learn a new markup language. want to fix something in the README? sorry, you have to learn a new markup language. want to fix a module in the standard library? sorry, you have to learn a new markup language. it's easy. there are only a couple of 100ks of specifications to read and understand. (that's only slightly larger than the Ruby language reference, and we're convinced that you'd rather learn another markup language than another programming language, right?) (and while you're at it, get a new keyboard; we don't care much about people using non-US keyboards...) ::: -1. the world doesn't need another markup language. there is only one markup language that everyone knows, and it's called HTML. the javadoc folks got it right. this one's all wrong. From mal@lemburg.com Wed Apr 3 10:50:58 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 03 Apr 2002 12:50:58 +0200 Subject: [Python-Dev] Deprecating C APIs (Use of PyArg_NoArgs()) References: <3C9F4AC0.C076679B@metaslash.com> <3C9F8B0F.D921E530@metaslash.com> <200203252052.g2PKqJa08509@pcp742651pcs.reston01.va.comcast.net> <3C9FA9D1.DA46BE04@metaslash.com> <200203252300.g2PN0DY09544@pcp742651pcs.reston01.va.comcast.net> <3CA9777F.BB3DA0DD@lemburg.com> <200204021437.g32Eb7C09097@pcp742651pcs.reston01.va.comcast.net> <3CAA0290.24D1E302@lemburg.com> <200204022104.g32L4pC20305@odiug.zope.com> Message-ID: <3CAADE92.3612E539@lemburg.com> "Martin v. Loewis" wrote: > > Guido van Rossum writes: > > > That's what I wanted, but now that you mention it I'm not sure if the > > warning module can currrently detect from which module a particular C > > API is called -- it'll probably put the blame on a particular line in > > the user's code, and then repeat the warning for each *different* line > > of user code. > > What *could* be done, of course, is to give the warning at most once. If that's possible, I'm +1 on generating a warning for the first usage of PyArg_NoArgs(). Would be nice to have the warning include the calling module name -- don't know whether that's possible, though (I have next to no experience with the warning framework). -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From skip@pobox.com Wed Apr 3 10:55:11 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 3 Apr 2002 04:55:11 -0600 Subject: [Python-Dev] Suggested memory API rules for 2.3 In-Reply-To: References: Message-ID: <15530.57231.202946.180190@12-248-41-177.client.attbi.com> Tim> Guido and I sat down and hashed this out Tuesday afternoon. Here's Tim> what I think Revealed Truth was. Hallelujah! Even I understood it. I can't imagine adapting to this scheme should be difficult. The only thing that threw me a little was the alphabet soup in Use as Function Designators Programs shall not use use a name from line B, C, D, F, G or H in a C context requiring a function designator. Names from lines A and E may be used in C contexts requiring a function designator. A function designator is a C expression having function type; C contexts requiring a function designator include as an actual argument to a function taking a function argument. This just means that PyMem_{Malloc, Realloc, Free} and PyObject_{Malloc, Realloc, Free} will be implemented as functions or simple macros that expand to a single function call, and that the others can be all sorts of CPP gobbledygook, right? Skip From barry@barrys-emacs.org Wed Apr 3 11:03:44 2002 From: barry@barrys-emacs.org (Barry Scott) Date: Wed, 3 Apr 2002 12:03:44 +0100 Subject: [Python-Dev] RE: [Python-Div] A Hygienic Macro System in Python? Message-ID: <000801c1daff$38684e00$060210ac@private> -----Original Message----- From: Barry Scott [mailto:barry.alan.scott@ntlworld.com] Sent: 02 April 2002 19:34 To: martin@v.loewis.de; barry@barrys-emacs.org Subject: RE: [Python-Div] A Hygienic Macro System in Python? The problem is linking. I cannot link against python21.dll as it will not be on all target systems. Now I can use DELAYLOAD to get at functions in macros. But DELAYLOAD is disabled if a variable in a DLL is referenced, hence the _CHECK problem, also some things like Py_None, Py_True and Py_False cannot be used for the same reason. BArry -----Original Message----- From: martin@v.loewis.de [mailto:martin@v.loewis.de] Sent: 02 April 2002 06:54 To: barry@barrys-emacs.org Subject: Re: [Python-Dev] A Hygienic Macro System in Python? "Barry Scott" writes: > Consider: my code attempts to load pythonXX.dll then if it is present > I find all the procedures that I need. I cannot use MACROS with implicit > references to functions, it will prevent me linking my app. > > All the _Check macros cannot be used. I would like a C function. Thanks, this is all I wanted to know. It is the _Check macros, and INCREF, DECREF you are after. Thanks for clarifying. I still don't understand the problem, though. For CXX, you do use the preprocessor, right? So why can't it just expand these macros? Furthermore, you say that you cannot use macros with implicit references to functions. Neither the _Check macros, nor INCREF/DECREF have implicit references to functions... Regards, Martin From barry@barrys-emacs.org Wed Apr 3 11:03:45 2002 From: barry@barrys-emacs.org (Barry Scott) Date: Wed, 3 Apr 2002 12:03:45 +0100 Subject: [Python-Dev] A Hygienic Macro System in Python? Message-ID: <000901c1daff$396e9a70$060210ac@private> python-dev@python.org -----Original Message----- From: Barry Scott [mailto:barry.alan.scott@ntlworld.com] Sent: 02 April 2002 02:09 To: martin@v.loewis.de Subject: RE: [Python-Dev] A Hygienic Macro System in Python? Martin, I'm quoting against the 2.1 code base. My code works against 2.2, but I'm sure these macros will break my code in mysterios ways in the future. My app is embedding Python within it. You can see my code in Python CXX connection on sourceforge. Consider: my code attempts to load pythonXX.dll then if it is present I find all the procedures that I need. I cannot use MACROS with implicit references to functions, it will prevent me linking my app. All the _Check macros cannot be used. I would like a C function. The INCREF/DECREF family cannot be used and worse these change definition on various #ifdef's, which I have to supply matching implementations for to allow debug vs. release builds. If you only use functions from the python DLL then delay loading can be used on windows. But there are all those unencapsulated variables that prevent this in a not trival app. They need a Get/Set interface. BArry -----Original Message----- From: martin@v.loewis.de [mailto:martin@v.loewis.de] Sent: 02 April 2002 00:35 To: barry@barrys-emacs.org Cc: python-dev@python.org Subject: Re: [Python-Dev] A Hygienic Macro System in Python? "Barry Scott" writes: > I need to be able to dynamically load python in my > application. Python integration is optional. I can look use the > GetProcAddress to get a real functions, but I have to know the > implementation inside the macros and duplicate that code. If there > are a pure procedural interface it would be a lot easier for my type > of app. Which macros would you like to avoid specifically? Regards, Martin From mal@lemburg.com Wed Apr 3 11:06:10 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 03 Apr 2002 13:06:10 +0200 Subject: [Python-Dev] Deprecating C APIs (Use of PyArg_NoArgs()) References: <3C9F4AC0.C076679B@metaslash.com> <3C9F8B0F.D921E530@metaslash.com> <200203252052.g2PKqJa08509@pcp742651pcs.reston01.va.comcast.net> <3C9FA9D1.DA46BE04@metaslash.com> <200203252300.g2PN0DY09544@pcp742651pcs.reston01.va.comcast.net> <3CA9777F.BB3DA0DD@lemburg.com> <200204021437.g32Eb7C09097@pcp742651pcs.reston01.va.comcast.net> <3CAA0290.24D1E302@lemburg.com> <20020402194625.GA28632@panix.com> Message-ID: <3CAAE222.8D55F2A5@lemburg.com> Aahz wrote: > > On Tue, Apr 02, 2002, M.-A. Lemburg wrote: > > > > I seriously think that we should install a set of common third party > > extensions somewhere to that we can easily check usage of the APIs in > > the outside world, e.g. take the packages Aahz collected recently and > > unzip them to some directory on python.org. A grep would then quickly > > produce proof of "common" usage. > > In case anyone wants to take up MAL's idea, here's an alphabetical list > of all the extensions I recognized in that thread: > > 4Suite > calldll > Gadfly > HTMLgen > mxODBC > NumPy > piddle > PIL > PyGame > PyOpenGL > PythonCard > SWIG > win32all > wxPython > ZODB Thanks, I'll setup a source archive directory on python.org. Hmm, you don't happen have a similar list of URLs for the source archives... :-) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From fredrik@pythonware.com Wed Apr 3 11:07:11 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 3 Apr 2002 13:07:11 +0200 Subject: [Python-Dev] Deprecating C APIs (Use of PyArg_NoArgs()) References: <3C9F4AC0.C076679B@metaslash.com><3C9F8B0F.D921E530@metaslash.com><200203252052.g2PKqJa08509@pcp742651pcs.reston01.va.comcast.net><3C9FA9D1.DA46BE04@metaslash.com><200203252300.g2PN0DY09544@pcp742651pcs.reston01.va.comcast.net><3CA9777F.BB3DA0DD@lemburg.com><200204021437.g32Eb7C09097@pcp742651pcs.reston01.va.comcast.net><3CAA0290.24D1E302@lemburg.com><200204022104.g32L4pC20305@odiug.zope.com> Message-ID: <03ca01c1daff$b5b3f530$ced241d5@hagrid> Martin v. Loewis wrote: > > > That's what I wanted, but now that you mention it I'm not sure if the > > warning module can currrently detect from which module a particular C > > API is called -- it'll probably put the blame on a particular line in > > the user's code, and then repeat the warning for each *different* line > > of user code. > > What *could* be done, of course, is to give the warning at most once. and what good is that, really? why would the *users* of a module I've written have to care about this? the Py_DEPRECATED approach (and the "grep" tool) is a much better way to use everybody's time. From ping@lfw.org Wed Apr 3 12:19:05 2002 From: ping@lfw.org (Ka-Ping Yee) Date: Wed, 3 Apr 2002 06:19:05 -0600 (CST) Subject: [Python-Dev] PEP 287: reStructuredText Standard DocstringFormat In-Reply-To: <013901c1daf0$e04e2b30$ced241d5@hagrid> Message-ID: On Wed, 3 Apr 2002, Fredrik Lundh wrote: > > -1. the world doesn't need another markup language. there is > only one markup language that everyone knows, and it's called > HTML. the javadoc folks got it right. this one's all wrong. I'm not against a new markup language, but i do feel that the specification of the language is just too big. What's with the 32 different section title adornment characters, the optional overline, and unspecified title styles (order "as encountered")? And that's just section titles. Do we really need five kinds of lists? How about the 15 different ways to number lists? The five ways to do hyperlink targets? And that's not all... I can appreciate the desire to come up with something flexible, but this goes too far for my taste. The current specification is about 10000 words; get it down to about 1000 and i might go for it. -- ?!ng From fdrake@acm.org Wed Apr 3 13:09:07 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 3 Apr 2002 08:09:07 -0500 Subject: [Python-Dev] New canonical location for PEPs In-Reply-To: References: <15530.31727.557290.92540@grendel.zope.com> Message-ID: <15530.65267.644411.998200@grendel.zope.com> Martin v. Loewis writes: > I wonder whether some 'bot could watch the checkin messages, and > update the PEPs online if it sees a checkin message for the PEPs. I'd be fine with that approach. Feel free to write the 'bot. My one concern with this would be involving the mail with this. python-checkins can be high-volume at times; running a 'bot that looked at each message might be too much, and it could easily end up running the several copies of the update if not careful, at the time when it does the least good (lots of changes going in 'cause Barry's cleaning out his PEPmeister mailbox). Getting this to work right could be more code than it's worth. Ok, ok, I guess I'm not fine with that approach. Feel free to prove me wrong, though. ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From mal@lemburg.com Wed Apr 3 13:22:08 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 03 Apr 2002 15:22:08 +0200 Subject: [Python-Dev] Deprecating C APIs (Use of PyArg_NoArgs()) References: <3C9F4AC0.C076679B@metaslash.com> <3C9F8B0F.D921E530@metaslash.com> <200203252052.g2PKqJa08509@pcp742651pcs.reston01.va.comcast.net> <3C9FA9D1.DA46BE04@metaslash.com> <200203252300.g2PN0DY09544@pcp742651pcs.reston01.va.comcast.net> <3CA9777F.BB3DA0DD@lemburg.com> <200204021437.g32Eb7C09097@pcp742651pcs.reston01.va.comcast.net> <3CAA0290.24D1E302@lemburg.com> <200204022104.g32L4pC20305@odiug.zope.com> Message-ID: <3CAB0200.4ACD6668@lemburg.com> Guido van Rossum wrote: > > > > > I'd suggest to have a PEP which documents all C API changes > > > > or deprecations together with a short decsription of how to > > > > update old code. > > > > > > It shouldn't be a PEP (that's a *proposal*). It should be part of > > > the C API docs, possibly an appendix. > > > > I was thinking of an informational PEP -- mostly because PEPs > > are easily accessible and also simple to write (LaTeX is not > > everybody's friend ;-). > > Yeah, that's fine. Let whoever writes it decide on the format. :-) Good idea .. as long as someone does :-) > > True, but there are other instances where this was a problem > > in the past, e.g. > > > > #if PY_VERSION_HEX < 0x01060000 > > > > #if !defined(PyObject_DEL) > > # define PyObject_DEL(x) free(x) > > # define PyObject_Del(x) free(x) > > #endif > > > > #endif > > Why the PY_VERSION_HEX guard? Isn't the !defined() guard enough? Well, yes... this is more to document when the change happened. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From marklists@mceahern.com Wed Apr 3 13:58:22 2002 From: marklists@mceahern.com (Mark McEahern) Date: Wed, 3 Apr 2002 07:58:22 -0600 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: Message-ID: [Christian Tanzer] > For me, Python's (old) way of handling boolean conditions is one of > its great strengths. This wasn't always the case. When I started > using Python, I was slightly put off by the lack of a boolean data > type. But quite quickly I learned to appreciate Python's way of > dealing with this issue. How does the PEP change "Python's (old) way of handling boolean conditions"? Correct me if I'm wrong, but under the PEP, you can still do this: l = range(10) if l: print "My, the list is not empty!" So what's different? // mark From mal@lemburg.com Wed Apr 3 14:03:45 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 03 Apr 2002 16:03:45 +0200 Subject: [Python-Dev] Re: PEP 285: Adding a bool type References: <2zRcLUAVjXp8EwVv@jessikat.fsnet.co.uk> <200204021340.g32Dekrq025410@ratthing-b246.strakt.com> <3CA9FD45.E6E33129@lemburg.com> <200204022145.g32LjAK20525@odiug.zope.com> Message-ID: <3CAB0BC1.20B90FED@lemburg.com> Guido van Rossum wrote: > > > I haven't followed this thread for quite some time, but since the > > PEP still seems alive, > > :-) > > > let me add some experience I've had with > > using the already existing singletons (Py_True and Py_False) > > in Python for recognizing truth values. > > > > Py_True and Py_False can be exposed in Python via > > > > True = (1==1) > > False = (1!=1) > > Yuck. What a horrible implementation detail. No really: comparisons should return booleans. > > and most comparison operations and quite a few other APIs > > returning truth values make use of these singletons. > > This works purely by accident. Given that at least Py_False is also > known as &_Py_ZeroStruct, I'm surprised that the code that optimizes > ints doesn't initialize small_ints[NSMALLNEGINTS] with Py_False. Py_True and Py_False are statically allocated objects which don't live on the heap. All the ints in small_ints do, however. _Py_ZeroStruct is probably just a naming inconsistency (Py_True maps to _Py_TrueStruct). > > I thought it would be a good idea to use those two > > singletons for protocols which use booleans such as > > XML-RPC. My experiences with this approach are, well, > > not so good :-/ > > > > The reason is that True and False are really integers > > and not of a special new type. Now they are singletons, > > which is good, since they represent the only states > > a boolean can have and in many cases work reasonably > > well as boolean representative, but not always > > (e.g. arithmetic operations such as True - True == False). > > Also, when trying to recognize the two singletons you have > > to use the "is" comparison -- "==" will fail to > > differentiate between 1 and True... > > > > def isboolean(x): > > return x in (True, False) > > > > ...doesn't work... > > > > def isboolean(x): > > return (x is True) or (x is False) > > > > ..does. > > The correct way if PEP 285 is accepted would of course be > isinstance(x, bool). Let's say, that's the goal. PEP 285 is not necessarily the complete answer to the problem ;-) > > As a conclusion, I think it would be better to make bool() a > > new type which does *not* inherit from integers, but which > > does know how deal with other types which are commonly > > used together with booleans such as integers. However, the > > type should implement boolean algebra and not try to > > mimic integer arithemtic, i.e. True - True raises an > > exception. > > That would break more code. E.g. (a!=0) + (b!=0) + (c!=0) counts how > many of (a, b, c) are nonzero; this would break. Right and this is goodness, since your example is plain wrong if seen as boolean operation. Boolean algebra doesn't have +, -, *, / operations. The right way to write this would be: int(a!=0) + int(b!=0) + int(c!=0) since you are counting values rather than finding truth and then only if we define int(True) := 1 and int(False) := 0. > > Py_True and Py_False should then be made singletons > > of this new type (possibly after a transition phase which > > replaces the singletons with a version that doesn't raise > > exceptions but instead issues warnings). > > That's what the PEP does (but with a different type). Right, but my point is that if you want to make booleans first class objects in Python, you should consider making them behave as defined in the text books rather than trying to make them feel like integers but look like truth values. > > This may sound painful at first, but in the long run, > > I believe, it'll result in the same benefits as other > > painful changes have or will (e.g. the change from integer > > division to floating point division). > > I don't see the introduction of the bool type as painful at all. > > I find getting it accepted painful though. :-( How about adding a new pure bool type to the language *without* changing any of the old Py_False/Py_True code (at first) ?! Then leave the code in for one release to see how people feel about it in real life and enable a warning for incorrect usage of comparison arithmetic in a second release. The third release would then enable the new logic per default. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From fredrik@pythonware.com Wed Apr 3 14:17:47 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 3 Apr 2002 16:17:47 +0200 Subject: [Python-Dev] Re: PEP 285: Adding a bool type References: Message-ID: <016701c1db1a$569543e0$0900a8c0@spiff> Mark McEahern wrote: > So what's different? more than one way to do it. more than one concept to explain to newcomers. more than one "obviously correct" way to do things. less is more. etc. From guido@python.org Wed Apr 3 14:26:12 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 03 Apr 2002 09:26:12 -0500 Subject: [Python-Dev] Re: PEP 279 In-Reply-To: Your message of "Wed, 03 Apr 2002 17:14:29 +1200." <200204030514.RAA00472@s454.cosc.canterbury.ac.nz> References: <200204030514.RAA00472@s454.cosc.canterbury.ac.nz> Message-ID: <200204031426.g33EQCd06837@pcp742651pcs.reston01.va.comcast.net> > Guido: > > > 0. itemize > > 1. enumerate [Greg E] > I'd like to add: > > 2. number -1. It looks like a noun to me. --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik@pythonware.com Wed Apr 3 14:22:25 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 3 Apr 2002 16:22:25 +0200 Subject: [Python-Dev] Re: PEP 285: Adding a bool type References: <2zRcLUAVjXp8EwVv@jessikat.fsnet.co.uk> <200204021340.g32Dekrq025410@ratthing-b246.strakt.com> <3CA9FD45.E6E33129@lemburg.com> <200204022145.g32LjAK20525@odiug.zope.com> <3CAB0BC1.20B90FED@lemburg.com> Message-ID: <016d01c1db1a$fc335620$0900a8c0@spiff> mal wrote: > > How about adding a new pure bool type to the language > *without* changing any of the old Py_False/Py_True > code (at first) ?! >=20 > Then leave the code in for one release to see how > people feel about it in real life and enable a warning > for incorrect usage of comparison arithmetic in a second=20 > release. The third release would then enable the new > logic per default. from __future__ import boolean ? (maybe it's time to introduce a __yagni__ module for people who prefer Harlin over van der Rohe? ;-) From guido@python.org Wed Apr 3 14:37:02 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 03 Apr 2002 09:37:02 -0500 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: Your message of "Wed, 03 Apr 2002 16:03:45 +0200." <3CAB0BC1.20B90FED@lemburg.com> References: <2zRcLUAVjXp8EwVv@jessikat.fsnet.co.uk> <200204021340.g32Dekrq025410@ratthing-b246.strakt.com> <3CA9FD45.E6E33129@lemburg.com> <200204022145.g32LjAK20525@odiug.zope.com> <3CAB0BC1.20B90FED@lemburg.com> Message-ID: <200204031437.g33Eb2506920@pcp742651pcs.reston01.va.comcast.net> > Right, but my point is that if you want to make booleans first class > objects in Python, you should consider making them behave as defined > in the text books rather than trying to make them feel like integers > but look like truth values. I reject this reasoning. Python is not trying to follow the text books. (In fact, maybe I should say "Python is trying not to follow the text books.") --Guido van Rossum (home page: http://www.python.org/~guido/) From marklists@mceahern.com Wed Apr 3 14:46:02 2002 From: marklists@mceahern.com (Mark McEahern) Date: Wed, 3 Apr 2002 08:46:02 -0600 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: <016701c1db1a$569543e0$0900a8c0@spiff> Message-ID: [Fredrik Lundh] > more than one way to do it. more than one concept to > explain to newcomers. more than one "obviously correct" > way to do things. less is more. etc. Hmm, I'm not sure I follow. This is Python without the PEP: falseValues = [{}, [], 0, None] for x in falseValues: if not x: print "%s is 'false'." % x This is Python with the PEP: falseValues = [{}, [], 0, None, False] for x in falseValues: if not x: print "%s is 'false'." % x I don't see this vast conceptual chasm between pre and post PEP that you appear to see. Please help me understand. Would the above be clearer if I called the collection of false values 'nothingValues' instead? I, personally, don't think so. // mark From vmarangozov@optimay.com Wed Apr 3 14:50:19 2002 From: vmarangozov@optimay.com (Marangozov, Vladimir (Vladimir)) Date: Wed, 3 Apr 2002 16:50:19 +0200 Subject: [Python-Dev] pymalloc killer Message-ID: <464156E0633B154091731247D911C9CF0E0CF5@ge0057excuag01.ags.agere.com> [Tim, in obmalloc.c] > #define ADDRESS_IN_RANGE(P, I) \ > ((I) < narenas && (uptr)(P) - arenas[I] < (uptr)ARENA_SIZE) Bravo! Very nice. Now that the pool header is reduced in size by one slot, I can only suggest a small optimization to get rid off the multiplication when trying to extend the free list. Since you know the code by heart already, this should be self explanatory. It's only 3 lines of code: 1) add a member in the pool struct, after capacity: uint lastoffset; /* free list tail block offset */ 2) In malloc, in the "try to extend the free list" block: size <<= ALIGNMENT_SHIFT; /* block size */ pool->lastoffset += size; pool->freeblock = (block *)pool + pool->lastoffset; 3) In the "init_pool" block: pool->capacity = ... pool->lastoffset = POOL_OVERHEAD + size; UNLOCK(); In other words, the lastoffset acts as an upper bound watermark. I didn't want to do that optimization before, because it would have resulted in a bigger pool header and waste of space. Now it's ok. Cheers, Vladimir From fredrik@pythonware.com Wed Apr 3 15:08:03 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 3 Apr 2002 17:08:03 +0200 Subject: [Python-Dev] Re: PEP 285: Adding a bool type References: Message-ID: <02a201c1db21$5c8ddfd0$0900a8c0@spiff> mark wrote: > Hmm, I'm not sure I follow. >=20 > This is Python without the PEP: >=20 > falseValues =3D [{}, [], 0, None] > for x in falseValues: > if not x: > print "%s is 'false'." % x >=20 > This is Python with the PEP: >=20 > falseValues =3D [{}, [], 0, None, False] > for x in falseValues: > if not x: > print "%s is 'false'." % x >=20 > I don't see this vast conceptual chasm between pre and post PEP that = you > appear to see. Please help me understand. can you explain how your second example covers everything that's in the PEP? can you explain why the second version is "better", in any sense of that word? From nas@python.ca Wed Apr 3 15:16:19 2002 From: nas@python.ca (Neil Schemenauer) Date: Wed, 3 Apr 2002 07:16:19 -0800 Subject: [Python-Dev] Re: Suggested memory API rules for 2.3 In-Reply-To: ; from tim.one@comcast.net on Wed, Apr 03, 2002 at 03:20:04AM -0500 References: Message-ID: <20020403071619.A369@glacier.arctrix.com> Tim Peters wrote: > Guido and I sat down and hashed this out Tuesday afternoon. Here's what I > think Revealed Truth was. Very good. Spelling out the rules was long overdue. > Note that the eight ways to spell "free" all have to map to the pymalloc > free when pymalloc is enabled in 2.3. There is no way to spell "give me the > raw platform free(), damn it", except for "free". I guess we need to do something special for pymalloc free() if PYMALLOC_DEBUG is defined. > We don't promise anything about the internals of the object allocator. Good. Someone might write platform specific versions that use mmap or something. > Recommended Practice > > When backward compatibility is not a concern, using this subset of > the raw memory and object memory APIs is recommended: > > PyMem_{Malloc, Realloc, Free} > PyObject_{Malloc, Realloc, Free} > PyObject_{New, NewVar} This should probably come up front in the documentation. All the rest is historical crap. :-) Neil From python@rcn.com Wed Apr 3 15:34:51 2002 From: python@rcn.com (Raymond Hettinger) Date: Wed, 3 Apr 2002 10:34:51 -0500 Subject: [Python-Dev] Re: PEP 279 References: <002d01c1da07$434c88c0$4c66accf@othello> <200204021357.g32DvTj06129@pcp742651pcs.reston01.va.comcast.net> Message-ID: <004101c1db25$197119c0$4066accf@othello> ----- Original Message ----- From: "Guido van Rossum" > 0. itemize > 1. enumerate After the conversation on names, are you ready to choose between enumerate(), enum(), and itemize()? enumerate -- I like this one best because it does what it says. Liked by most of the py-dev respondants. enum -- I like this one too. A tiny bit of clarity traded for pithiness (like dict vs dictionary). Must rhyme with 'doom', not with 'dumb' . itemize -- Could be confusing since amap.items() != list(itemize(amap)). Unfortunate association with income taxes :( Raymond Hettinger, CPA From marklists@mceahern.com Wed Apr 3 15:55:48 2002 From: marklists@mceahern.com (Mark McEahern) Date: Wed, 3 Apr 2002 09:55:48 -0600 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: <02a201c1db21$5c8ddfd0$0900a8c0@spiff> Message-ID: [me] > This is Python with the PEP: > > falseValues = [{}, [], 0, None, False] > for x in falseValues: > if not x: > print "%s is 'false'." % x [Fredrik Lundh] > can you explain how your second example covers > everything that's in the PEP? It doesn't cover everything in the PEP. Here's how I think of the PEP: With this PEP, we get to have our cake and eat it too: 1. There is one obviously correct way to return a truth value. 2. Polymorphous perversity with respect to truth still works. What's everybody complaining about? Cheers, // mark From ark@research.att.com Wed Apr 3 15:59:13 2002 From: ark@research.att.com (Andrew Koenig) Date: 03 Apr 2002 10:59:13 -0500 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: <016701c1db1a$569543e0$0900a8c0@spiff> References: <016701c1db1a$569543e0$0900a8c0@spiff> Message-ID: >> So what's different? Fredrik> more than one way to do it. more than one concept to Fredrik> explain to newcomers. more than one "obviously correct" Fredrik> way to do things. less is more. etc. I'm a little confused, because it seems to me that Python already has more than one way to do this particular thing. Specifically: If I want to write a function that answers a yes/no question, I have lots of possible ways of spelling yes (1, 2, "yes", and so on) and lots of possible ways of spelling no (0, {}, None, and so on). There isn't a single preferred way. -- Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark From tree@basistech.com Wed Apr 3 16:14:00 2002 From: tree@basistech.com (Tom Emerson) Date: Wed, 3 Apr 2002 11:14:00 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format In-Reply-To: References: Message-ID: <15531.10824.991521.687697@magrathea.basistech.com> David Goodger writes: [...] > Proposed alternatives have included: > > - XML [3]_, SGML [4]_, DocBook [5]_, HTML [6]_, XHTML [7]_ > > XML and SGML are explicit, well-formed meta-languages suitable > for all kinds of documentation. XML is a variant of SGML. They > are best used behind the scenes, because they are verbose, > difficult to type, and too cluttered to read comfortably as > source. DocBook, HTML, and XHTML are all applications of SGML > and/or XML, and all share the same basic syntax and the same > shortcomings. And how much of HTML is required for marking up documentation? JavaDoc doesn't appear to suffer from this problem. I've never heard this argument given by programmers who are writing documentation. I use HTML in my JavaDoc comments. I use HTML in my Doxygen comments (in C++). Why not use HTML in my Python comments? DocBook is much more verbose, as it concentrates almost exclusively on semantics, not display. It is overkill for this. XML and SGML are by themselves not an alternative: they are a means to the end. XML is an SGML application. DocBook is an XML and/or SGML application. HTML is an SGML application. XHTML is an XML application. > - JavaDoc [10]_ > > Special comments before Java classes and functions serve to > document the code. A program to extract these, and turn them > into HTML documentation is called javadoc, and is part of the > standard Java distribution. However, the only output format > that is supported is HTML, and JavaDoc has a very intimate > relationship with HTML, using HTML tags for most markup. [...] This is patently false: there are Doclets available that convert to a wide variety of formats. Sun provides a MIF doclet, and third parties have provided doclets for RTF, TexInfo, LaTeX, and DocBook. There is very little that cannot be marked up in HTML that cannot be converted to other formats in a straight forward way. I'll raise Doxygen as another example: the comments utilize HTML (though you can escape for specific processor features (e.g., TeX equations) if necessary) and documentation can be generated in HTML, RTF, PostScript, Hyperlinked PDF, compressed HTML, and Unix man page format. You also forget to mention TexInfo, one of the older and more widely used documentation format for programmer docs. I don't have a vote, but if I did, -1. -tree -- Tom Emerson Basis Technology Corp. Sr. Computational Linguist http://www.basistech.com "Beware the lollipop of mediocrity: lick it once and you suck forever" From fredrik@pythonware.com Wed Apr 3 16:16:17 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 3 Apr 2002 18:16:17 +0200 Subject: [Python-Dev] Re: PEP 285: Adding a bool type References: <016701c1db1a$569543e0$0900a8c0@spiff> Message-ID: <00af01c1db2a$e7316d10$ced241d5@hagrid> Andrew Koenig wrote: > Specifically: If I want to write a function that answers a yes/no > question, I have lots of possible ways of spelling yes (1, 2, "yes", > and so on) and lots of possible ways of spelling no (0, {}, None, > and so on). There isn't a single preferred way. if you study real python code, you'll find that there is. From ark@research.att.com Wed Apr 3 16:19:33 2002 From: ark@research.att.com (Andrew Koenig) Date: Wed, 3 Apr 2002 11:19:33 -0500 (EST) Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: <00af01c1db2a$e7316d10$ced241d5@hagrid> (fredrik@pythonware.com) References: <016701c1db1a$569543e0$0900a8c0@spiff> <00af01c1db2a$e7316d10$ced241d5@hagrid> Message-ID: <200204031619.g33GJXm02577@europa.research.att.com> >> Specifically: If I want to write a function that answers a yes/no >> question, I have lots of possible ways of spelling yes (1, 2, "yes", >> and so on) and lots of possible ways of spelling no (0, {}, None, >> and so on). There isn't a single preferred way. Fredrik> if you study real python code, you'll find that there is. I mean that the language doesn't prefer a single way, as opposed to programmers' conventions. From guido@python.org Wed Apr 3 16:19:14 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 03 Apr 2002 11:19:14 -0500 Subject: [Python-Dev] Re: PEP 279 In-Reply-To: Your message of "Wed, 03 Apr 2002 10:34:51 EST." <004101c1db25$197119c0$4066accf@othello> References: <002d01c1da07$434c88c0$4c66accf@othello> <200204021357.g32DvTj06129@pcp742651pcs.reston01.va.comcast.net> <004101c1db25$197119c0$4066accf@othello> Message-ID: <200204031619.g33GJEk21882@odiug.zope.com> > > 0. itemize > > 1. enumerate > > After the conversation on names, are you ready to choose between > enumerate(), enum(), and itemize()? > > enumerate -- I like this one best because it does what it says. Liked by > most of the py-dev respondants. This one gets my vote. > enum -- I like this one too. A tiny bit of clarity traded for pithiness > (like dict vs dictionary). Must rhyme with 'doom', not with 'dumb' . Too similar to enum in tons of other languages. > itemize -- Could be confusing since amap.items() != list(itemize(amap)). > Unfortunate association with income taxes :( I hear this over and over so I give up on it. IOW, enumerate() it is. When you send Barry your update to the PEP reflecting this, he can mark it Accepted. (I still find it strange to mark a PEP Accepted that has two parts, only one of which is accepted. But so be it. Next time be sure to make separate PEPs.) --Guido van Rossum (home page: http://www.python.org/~guido/) From jacobs@penguin.theopalgroup.com Wed Apr 3 16:31:03 2002 From: jacobs@penguin.theopalgroup.com (Kevin Jacobs) Date: Wed, 3 Apr 2002 11:31:03 -0500 (EST) Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: <00af01c1db2a$e7316d10$ced241d5@hagrid> Message-ID: On Wed, 3 Apr 2002, Fredrik Lundh wrote: > Andrew Koenig wrote: > > Specifically: If I want to write a function that answers a yes/no > > question, I have lots of possible ways of spelling yes (1, 2, "yes", > > and so on) and lots of possible ways of spelling no (0, {}, None, > > and so on). There isn't a single preferred way. > > if you study real python code, you'll find that there is. Having managed 200k+ LOC Python projects with at times 20+ developers, I find that there isn't. At least not without getting out a baseball bat. The first thing many of our programmers want to do at the top of their modules is to define: False = 0 True = not False or false = 0 true = 1 or TRUE = 1 FALSE = 0 or from my_local_trashcan_module import true, false or some fairly extreme object implementations in Python or C. Of course, I have to break their fingers and force them to stop re-inventing their own true and false literals, and to use 1's and 0's like the rest of the disadvantaged Python world. Ah, the joys of code review.... Believe it or not, this, naming conventions, and consistent exception handling styles are among the biggest bones of contention we have during our code review sessions. The moral of the story is that I will be extremely happy once there is a globally blessed way of spelling true and false other than 1 and 0. Of course, I and my team aren't exactly typical Python developers. Though wouldn't it be nice if we were? -Kevin -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com From barry@zope.com Wed Apr 3 16:35:38 2002 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 3 Apr 2002 11:35:38 -0500 Subject: [Python-Dev] Re: PEP 285: Adding a bool type References: <016701c1db1a$569543e0$0900a8c0@spiff> Message-ID: <15531.12122.601076.455659@anthem.wooz.org> >>>>> "AK" == Andrew Koenig writes: AK> Specifically: If I want to write a function that answers a AK> yes/no question, I have lots of possible ways of spelling yes AK> (1, 2, "yes", and so on) and lots of possible ways of spelling AK> no (0, {}, None, and so on). There isn't a single preferred AK> way. This is a strong pro (IMO) for PEP 285. When I code a flag, do I use 0 and 1, or None and 1, or ...what? When you see code that says "x.isthing = 1" what does that tell you about how this attribute will be used? I like the bool because it provides clarity when coding flag values, and you still maintain the (IMHO beautiful) feature of being able to say "this variable has a value, or it's never been set". I.e. flag=None, flag=True, flag=False. The strongest con for PEP 285 seems to be the natural inclination for people to want to write "if cond == True:". Heck, /I/ certainly wrote that the first time I thought about bools years ago (and even made that "work" in my early pure-Python prototypes). But it doesn't make sense since "if cond:" and "if not cond:" are better than comparison against the boolean constants. There's two reasons we teased out about why people may tend to want to compare against True and False. People may think that the comparison is somehow part of the syntax of the if statement. Also, using the comparison can sometimes be more aligned with English grammar. E.g. this reads well on its own: if year.isLeapYear(): Note the use of `is' which tends to be a common prefix on predicates. But if you don't use that prefix, then you are maybe inclined to add the test: if flag == True: so that it reads somewhat like "if flag is True". (Why don't we tend to write exactly that? Maybe because as Python programmers, we're trained not to use identity tests except in very limited situations?). -Barry From barry@zope.com Wed Apr 3 16:38:38 2002 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 3 Apr 2002 11:38:38 -0500 Subject: [Python-Dev] Re: PEP 279 References: <002d01c1da07$434c88c0$4c66accf@othello> <200204021357.g32DvTj06129@pcp742651pcs.reston01.va.comcast.net> <004101c1db25$197119c0$4066accf@othello> <200204031619.g33GJEk21882@odiug.zope.com> Message-ID: <15531.12302.1903.290116@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> (I still find it strange to mark a PEP Accepted that has two GvR> parts, only one of which is accepted. But so be it. Next GvR> time be sure to make separate PEPs.) I've added some language to PEP 1 to more strongly encourage focussed, smaller PEPs. I'll reserve the right in the future to reject PEPs that seem too broad. -Barry From ping@lfw.org Wed Apr 3 16:44:01 2002 From: ping@lfw.org (Ka-Ping Yee) Date: Wed, 3 Apr 2002 10:44:01 -0600 (CST) Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: <200204031619.g33GJXm02577@europa.research.att.com> Message-ID: Andrew Koenig wrote: > Specifically: If I want to write a function that answers a yes/no > question, I have lots of possible ways of spelling yes (1, 2, "yes", > and so on) and lots of possible ways of spelling no (0, {}, None, > and so on). There isn't a single preferred way. Fredrik Lundh wrote: > if you study real python code, you'll find that there is. Andrew Koenig wrote: > I mean that the language doesn't prefer a single way, > as opposed to programmers' conventions. This doesn't mean anything in practice. Every yes/no function i've encountered returns 1 for yes and 0 for no. (If it returns something else, then it really has more to say than yes or no.) In fact the language actually does prefer a particular way, since x == x returns 1. The addition of another type really does change the set of preferred truth values from {0, 1} to {0, False, 1, True}. The typical practicing Python programmer who wants to say "true" would have to decide -- for each instance -- whether to say "1" or "True". And the programmer cannot happily ignore the issue and stick to using 0 and 1, because if comparisons start to return True and False, dealing with a mix of four truth values is unavoidable. This PEP makes me uneasy because it causes me to see lots of extra casts in Python's future (not to mention confused beginners). -- ?!ng From akuchlin@mems-exchange.org Wed Apr 3 16:52:54 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Wed, 3 Apr 2002 11:52:54 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format In-Reply-To: <15531.10824.991521.687697@magrathea.basistech.com> References: <15531.10824.991521.687697@magrathea.basistech.com> Message-ID: <20020403165253.GA23796@ute.mems-exchange.org> On Wed, Apr 03, 2002 at 11:14:00AM -0500, Tom Emerson wrote: >And how much of HTML is required for marking up documentation? JavaDoc There's also IBTWSH, a DTD describing a small subset of HTML: It's intended for use in other DTDs that want slightly richer text than just plain text, but not all of HTML; for example, I use it in my quotation DTD. I keep meaning to write some sort of general IBTWSH processing code as a Python module. --amk (www.amk.ca) "Dorset Street -- the most evil in London, I'm told." -- Hinton, in FROM HELL #2 From mwh@python.net Wed Apr 3 16:53:57 2002 From: mwh@python.net (Michael Hudson) Date: 03 Apr 2002 17:53:57 +0100 Subject: [Python-Dev] Re: Suggested memory API rules for 2.3 In-Reply-To: Neil Schemenauer's message of "Wed, 3 Apr 2002 07:16:19 -0800" References: <20020403071619.A369@glacier.arctrix.com> Message-ID: <2mn0wkena2.fsf@starship.python.net> Neil Schemenauer writes: > Tim Peters wrote: > > Recommended Practice > > > > When backward compatibility is not a concern, using this subset of > > the raw memory and object memory APIs is recommended: > > > > PyMem_{Malloc, Realloc, Free} > > PyObject_{Malloc, Realloc, Free} > > PyObject_{New, NewVar} > > This should probably come up front in the documentation. All the rest > is historical crap. :-) We should probably provide an example of preprocessor hackery that lets you use these interfaces and remain backward compatible as far as possible. Maybe even a little .h file which we can suggest extension authors bundle with their extensions. Cheers, M. -- On the other hand, the following areas are subject to boycott in reaction to the rampant impurity of design or execution, as determined after a period of study, in no particular order: ... http://www.naggum.no/profile.html From guido@python.org Wed Apr 3 16:56:56 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 03 Apr 2002 11:56:56 -0500 Subject: [Python-Dev] Suggested memory API rules for 2.3 In-Reply-To: Your message of "Wed, 03 Apr 2002 03:20:04 EST." References: Message-ID: <200204031657.g33GuuH22110@odiug.zope.com> > "Programs" does not include the Python core. We can do anything we > need to do in the core, based on our complete knowledge of > each release's internal implementation details. However, since many people use the core as an example, we may have to be careful here. Certainly xxmodule.c, xxobject.c and xxsubtype.c should do the recommended thing; but if too many of the built-in objects use the macros despite those being in danger of deprecation, that might still perpetuate use of those macros. > Mixing and Matching > > After memory has been obtained via one of these functions, it > should be resized and freed only by a function from the same line, > except that PyMem_Free may be used freely in place of PyMem_Del, > and PyObject_Free in place of PyObject_Del. I actually had a slightly different rule in mind: mixing and matching between two adjacent lines that only differ by case is also permitted. Except of course that the macro lines will be deprecated, so maybe it doesn't matter to spell this out. > Maintaining the Free/Del pseudo-distinction is pointless. > > For backward compatibility, memory obtained via the object memory > family can be freed by any of Py{Mem, Object}_{Free, FREE, Del, DEL}. > Mixing functions from the object family with the raw memory family > is likely to become deprecated, > > Memory obtained by PyMem_{Malloc, MALLOC} shall be resized only > by PyMem_{Realloc, REALLOC}. > > Memory obtained by PyMem_{New, NEW} shall be resized only > by PyMem_{Resize, RESIZE}. > > Memory obtained by PyObject_{Malloc, MALLOC} shall be resized > only by PyObject_{Realloc, REALLOC}. > > Note that the eight ways to spell "free" all have to map to the > pymalloc free when pymalloc is enabled in 2.3. There is no way to > spell "give me the raw platform free(), damn it", except for "free". > If we think it's important to have a such a way in the core, it > should be added to the private API. IMO, PyMem_Malloc() and PyMem_Free() should be good enough. > Relationship to Platform Allocator > > All names in lines A, B, C and D ultimately invoke the platform > C malloc, realloc, or free. However, programs shall not mix any > of these names with direct calls to the platform malloc, calloc, > realloc, or free referencing the same base memory addresses, as > Python may need to perform bookkeeping of its own around its calls > to the platform allocator. Really? Why not just say these are wrappers around malloc and free? (On platforms where it matters, they will be guaranteed to use the same heap as the core uses -- this is apparently an issue with Windows DLLs.) --Guido van Rossum (home page: http://www.python.org/~guido/) From marklists@mceahern.com Wed Apr 3 16:59:49 2002 From: marklists@mceahern.com (Mark McEahern) Date: Wed, 3 Apr 2002 10:59:49 -0600 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: <15531.12122.601076.455659@anthem.wooz.org> Message-ID: [Barry A. Warsaw] > The strongest con for PEP 285 seems to be the natural inclination for > people to want to write "if cond == True:". I don't see why you want to blame that inclination on PEP 285. Folks are perfectly capable of monstrosities like this without it: x = isinstance(foo, bar) if x == 1: ... If, as Fredrik says, there's already one obvious way to return True (and that's the int 1), then we already have the problem that people will be inclined to compare something to this canonical truth in conditional statements. I suppose you're saying that PEP 285 makes it worse? I have no idea whether you're right. It certainly doesn't make ME more likely to do it. ;-) Cheers, // mark From fredrik@pythonware.com Wed Apr 3 17:03:24 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 3 Apr 2002 19:03:24 +0200 Subject: [Python-Dev] Re: PEP 285: Adding a bool type References: <016701c1db1a$569543e0$0900a8c0@spiff> <00af01c1db2a$e7316d10$ced241d5@hagrid> <200204031619.g33GJXm02577@europa.research.att.com> Message-ID: <016b01c1db31$8da51010$ced241d5@hagrid> andrew wrote: > > Fredrik> if you study real python code, you'll find that there is. > > I mean that the language doesn't prefer a single way, > as opposed to programmers' conventions. it's more than just a convention -- built-in functions and operators always return 0 for false and 1 for true whenever they need to in- vent a boolean value. this is documented in the language reference. (and if you look inside, there's something called Py_False that's an integer zero, and something called Py_True that is an integer one). this is also true for the standard library, and lots of major extensions. also note that the PEP says "The bool type would be a straightforward subtype (in C) of the int type, and the values False and True would behave like 0 and 1 in most respects". Guido didn't pick those values out of thin air... From marklists@mceahern.com Wed Apr 3 17:05:44 2002 From: marklists@mceahern.com (Mark McEahern) Date: Wed, 3 Apr 2002 11:05:44 -0600 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: Message-ID: [Kevin Jacobs] > The moral of the story is that I will be extremely happy once there is a > globally blessed way of spelling true and false other than 1 and 0. Of > course, I and my team aren't exactly typical Python developers. Though > wouldn't it be nice if we were? I'd like to consciously go out of my way and violate all rules of netiquette and say simply this: Me too. Cheers, // mark From marklists@mceahern.com Wed Apr 3 17:12:58 2002 From: marklists@mceahern.com (Mark McEahern) Date: Wed, 3 Apr 2002 11:12:58 -0600 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: Message-ID: [Ka-Ping Yee] > The addition of another type really does change the set of preferred > truth values from {0, 1} to {0, False, 1, True}. The typical practicing > Python programmer who wants to say "true" would have to decide -- for > each instance -- whether to say "1" or "True". Nope, the set of preferred truth values would be: True, False You could, of course, continue to use 1, 0 if you like. In the long run, why would you? > And the programmer cannot happily ignore the issue and stick to using > 0 and 1 [...] Of course she can. In the long run, why would she? > [...] because if comparisons start to return True and False, dealing > with a mix of four truth values is unavoidable. This PEP makes me uneasy > because it causes me to see lots of extra casts in Python's future (not > to mention confused beginners). Can you give me just one example of these supposedly necessary 'extra casts' that is not gratuitous and intentionally sloppy? I'm too dense to come up with one on my own. Hmm, this PEP makes me delighted because I see lots of clean code and happy, productive beginners. But, to each his own. Cheers, // mark From guido@python.org Wed Apr 3 17:12:36 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 03 Apr 2002 12:12:36 -0500 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: Your message of "Wed, 03 Apr 2002 10:44:01 CST." References: Message-ID: <200204031712.g33HCaP22228@odiug.zope.com> > The addition of another type really does change the set of preferred > truth values from {0, 1} to {0, False, 1, True}. The typical > practicing Python programmer who wants to say "true" would have to > decide -- for each instance -- whether to say "1" or "True". No. If they want to use the *recommended* values they should always use False or True. 0 and 1 are relegated to the domain of non-representative values of the set of false and true objects, just as None and "true" (the string literal). > And the programmer cannot happily ignore the issue and stick to > using 0 and 1, because if comparisons start to return True and > False, dealing with a mix of four truth values is unavoidable. This > PEP makes me uneasy because it causes me to see lots of extra casts > in Python's future (not to mention confused beginners). You misunderstand it. Strings, lists, numbers and so on are still acceptable as truth values, and when you want to know whether x is true or false, you still have to say "if x:" -- never "if x == True:". In addition, for backward compatibility 0 == False and 1 == True, so mixing them shouldn't be a problem if you know your values are chosen from the set {0, False, 1, True}. If you find yourself comparing an unknown value x to a known truth value to determine whether x is true or false, stop right there: you've *already* got a problem. If you don't have a problem because it works in practice (meaning x is known to be 0 or 1) the backwards compatibility of the PEP kicks in and saves your butt. This won't be deprecated: the PEP promises that False==0 and True==1 will always hold. (There's a question for reviewers about this, but consider it decided -- bools will always mix freely with ints.) --Guido van Rossum (home page: http://www.python.org/~guido/) From David Abrahams" Message-ID: <017c01c1db33$53e38800$1101a8c0@boostconsulting.com> FWIW, I do the same thing every time I need to work with booleans. I try to "say what I mean" when I program, and, I'm sorry, but "return 1|0" just doesn't say "I mean to return something which will be interpreted as true|false". I want the language to help me say what I mean. I do understand about the backward-compatibility issues introduced by adding bool, but I don't understand why there are claims that there will be more than one official way to return boolean results. It seems to me that the only thing pointing to 1|0 as the right way would be legacy code, and everything in the language would point at true|false. -Dave ----- Original Message ----- From: "Mark McEahern" > [Kevin Jacobs] > > The moral of the story is that I will be extremely happy once there is a > > globally blessed way of spelling true and false other than 1 and 0. Of > > course, I and my team aren't exactly typical Python developers. Though > > wouldn't it be nice if we were? > > I'd like to consciously go out of my way and violate all rules of netiquette > and say simply this: > > Me too. From ark@research.att.com Wed Apr 3 17:36:14 2002 From: ark@research.att.com (Andrew Koenig) Date: 03 Apr 2002 12:36:14 -0500 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: <016b01c1db31$8da51010$ced241d5@hagrid> References: <016701c1db1a$569543e0$0900a8c0@spiff> <00af01c1db2a$e7316d10$ced241d5@hagrid> <200204031619.g33GJXm02577@europa.research.att.com> <016b01c1db31$8da51010$ced241d5@hagrid> Message-ID: >> I mean that the language doesn't prefer a single way, >> as opposed to programmers' conventions. Fredrik> it's more than just a convention -- built-in functions and Fredrik> operators always return 0 for false and 1 for true whenever Fredrik> they need to invent a boolean value. this is documented in Fredrik> the language reference. On the other hand, the library isn't nearly so consistent. Just for fun, I started browsing the library documentation at random. The very first page I visited was the description of gc, the garbage collector interface. The first function that returns a truth value there is described as isenabled() Returns true if automatic collection is enabled. Some more examples: 2.2.6.1 String Methods endswith(suffix[,start[,end]]) Return true if the string ends with the specified suffix, otherwise return false. Similarly for isalnum, isalpha, isdigit, islower, isspace. 2.2.8 File Objects: isatty() Return true if the file is connected to a tty(-like) device, else false. 6.2 os.path -- Common pathname manipulations exists(path) Return true if path refers to an existing path. I'm sure there are others. -- Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark From guido@python.org Wed Apr 3 17:51:52 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 03 Apr 2002 12:51:52 -0500 Subject: [Python-Dev] Deprecating C APIs (Use of PyArg_NoArgs()) In-Reply-To: Your message of "Wed, 03 Apr 2002 12:50:58 +0200." <3CAADE92.3612E539@lemburg.com> References: <3C9F4AC0.C076679B@metaslash.com> <3C9F8B0F.D921E530@metaslash.com> <200203252052.g2PKqJa08509@pcp742651pcs.reston01.va.comcast.net> <3C9FA9D1.DA46BE04@metaslash.com> <200203252300.g2PN0DY09544@pcp742651pcs.reston01.va.comcast.net> <3CA9777F.BB3DA0DD@lemburg.com> <200204021437.g32Eb7C09097@pcp742651pcs.reston01.va.comcast.net> <3CAA0290.24D1E302@lemburg.com> <200204022104.g32L4pC20305@odiug.zope.com> <3CAADE92.3612E539@lemburg.com> Message-ID: <200204031751.g33Hpq907186@pcp742651pcs.reston01.va.comcast.net> > If that's possible, I'm +1 on generating a warning for the first > usage of PyArg_NoArgs(). Would be nice to have the warning include > the calling module name -- don't know whether that's possible, > though (I have next to no experience with the warning framework). It peeks in the current globals for __name__. The current globals are those of the most recent Python module on the stack. Since it also gives a line number, you might be able to track down which call it's referring to. But it might be somewhat surprising. --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@zope.com Wed Apr 3 17:45:15 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Wed, 3 Apr 2002 12:45:15 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format In-Reply-To: <20020403165253.GA23796@ute.mems-exchange.org> References: <15531.10824.991521.687697@magrathea.basistech.com> <20020403165253.GA23796@ute.mems-exchange.org> Message-ID: <15531.16299.945226.362456@slothrop.zope.com> I've never participated in the doc-sig, and I haven't given a lot of thought to writing complete documentation in docstrings. Given my limited interest, I have always been puzzled by why Python has taken so long to come up with some simple conventions for structuring docstrings. The only other example I was familiar with was JavaDoc. I've never written an JavaDoc, but I've browsed lots of HTML generated by JavaDoc. The one time I looked at JavaDoc, it struck me as quite simple; I felt confident that I could write it given my limited knowledge of and interest in HTML. It also appeared that JavaDoc had a limited feature set, which also seemed like a strength: Let's not write fancy, formatted reports in docstrings. I don't have the same confidence in reStructuredText. It looks like I can mostly read it, but it seems hard to learn how to write. It seems burdensome to require module authors to learn Python and reStructuredText just to contribute to the std library. Jeremy From martin@v.loewis.de Wed Apr 3 18:00:30 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 03 Apr 2002 20:00:30 +0200 Subject: [Python-Dev] New canonical location for PEPs In-Reply-To: <20020403004949.I16962@tummy.com> References: <15530.31727.557290.92540@grendel.zope.com> <20020402233722.H16962@tummy.com> <20020403004949.I16962@tummy.com> Message-ID: Sean Reifschneider writes: > On Wed, Apr 03, 2002 at 09:28:17AM +0200, Martin v. Loewis wrote: > >really hard to get this mechanism to update web pages on a different > >machine (especially under a different account than the committer's > > That's what ssh with public/private key pairs and "command=" in > authorized_keys is for. :-) Yes, I know - while it solves this aspect of the problem, it isn't really simple, either. > There is some reasonable mechanism for doing this, as it *IS* done in some > places. The guy who had set it up for one web site and was therefore > supposed to be helping get it set up on a local LUG web site has thus far > not done so. :-( It may be that he has done something *like* you propose. However, I still doubt that you could do *exactly* that. Regards, Martin From ping@lfw.org Wed Apr 3 18:06:31 2002 From: ping@lfw.org (Ka-Ping Yee) Date: Wed, 3 Apr 2002 12:06:31 -0600 (CST) Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: <200204031712.g33HCaP22228@odiug.zope.com> Message-ID: On Wed, 3 Apr 2002, Guido van Rossum wrote: > You misunderstand it. Strings, lists, numbers and so on are still > acceptable as truth values, and when you want to know whether x is > true or false, you still have to say "if x:" -- never "if x == True:". Yes yes. I understand this part just fine. It's not the *listening* i'm concerned about -- 'if' takes care of that easily for me. It hears [], None, 0 as false and hears 'spam', {1: 2}, 47.3 as true. It's the *speaking*. When i want to say true or false, then there's the dilemma. I know, your answer is "you should always just say True or False", and Mark McEahern said the same thing. But this cannot be so in practice: *everything* already returns 0 or 1. (It may be possible to get around this if we commit to changing the entire standard library before releasing a version of Python with True and False, but alas, this is not the only issue... read on.) As long as True and False are somewhere represented as 0 and 1, the values 0 and 1 will never lose legitimacy as booleans. This business with str() and/or repr() producing "0" or "1" for backwards compatibility prevents us from considering 0 and 1 relegated to a truly non-boolean status. Consider this: >>> a = [0, False, 1, True] >>> print a [0, 0, 1, 1] >>> for x in a: print x 0 0 1 1 Good heavens! What about this: >>> d = {} >>> d[0] = 'zero' >>> d[False] = 'false' >>> len(d) 1 or 2? Basically, the concept of having a permanently schizophrenic type in the language scares me. The above shows, i believe, that a reasonable implementation must print True as True and False as False, and never mention 1 or 0. Moreover, as soon as you start sorting a bag of objects, or keying dictionaries on objects, you are forced to run into the distinction between 0 and False, and between 1 and True. I'm not against the idea of booleans, of course -- but i do think that halfway booleans are worse than what we have now. And getting to real booleans [*] involves real pain; it's just a question of whether that pain is worth it. Even if we get all the way there -- as in we manage to convert enough code and convince everyone to use the new style -- i will never ever want "and" and "or" to return booleans (i just hope that doesn't confuse anyone). -- ?!ng [*] By real booleans, i mean the following. (Booleans would have to behave like this for me to consider them "good enough" to be better than what we have now.) >>> False, repr(False), str(False) (False, 'False', 'False') >>> True, repr(False), str(False) (True, 'False', 'False') >>> False + True TypeError... >>> False == None 0 >>> False == 0 0 >>> True == 1 0 >>> {0: 0, False: False, 1: 1, True: True} {0: 0, False: False, 1: 1, True: True} ... and probably >>> None < False < True < 0 True (Hee hee -- i suppose the fact that "boolean" starts with a "b" gets us this for free. But i wonder how many people are going to be puzzled by True < 0?) From gsw@agere.com Wed Apr 3 18:08:13 2002 From: gsw@agere.com (Gerald S. Williams) Date: Wed, 3 Apr 2002 13:08:13 -0500 Subject: [Python-Dev] RE: Re: PEP 285: Adding a bool type In-Reply-To: Message-ID: Mark McEahern wrote: > If, as Fredrik says, there's already one obvious way to return True (and > that's the int 1), then we already have the problem that people will be > inclined to compare something to this canonical truth in conditional > statements. Actually, he wrote "preferred", not "obvious". I.e.: def bool(x): return not not x will return 0 or 1. but (), [], "", and None also evaluate as false in a logical context, while most other things evaluate as true. Having canonical True and False values would lead newcomers to compare to them. This would work, for example: DONE = y and (x > y) ... if DONE is False: ... On the other hand, this wouldn't (unless y is a boolean): DONE = (x > y) and y ... if DONE is False: ... Of course, it works either way if you replace "DONE is False" with "not DONE". Or if you replace "y" with "bool(y)". Since there aren't canonical True/False values, there is currently an obvious solution: use "not DONE". But "native" boolean support will lead to mistakes like this unless you start taking steps to introduce type-safety with operations involving boolean values. This is definitely going farther than was intended by the PEP. -Jerry -O Gerald S. Williams, 55A-134A-E : mailto:gsw@agere.com O- -O AGERE SYSTEMS, 6755 SNOWDRIFT RD : office:610-712-8661 O- -O ALLENTOWN, PA, USA 18106-9353 : mobile:908-672-7592 O- From marklists@mceahern.com Wed Apr 3 18:14:27 2002 From: marklists@mceahern.com (Mark McEahern) Date: Wed, 3 Apr 2002 12:14:27 -0600 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: Message-ID: [Ka-Ping Yee] > Consider this: > > >>> a = [0, False, 1, True] > > >>> print a > [0, 0, 1, 1] > > >>> for x in a: print x > 0 > 0 > 1 > 1 Actually, if you use the sample code provided by the PEP and run this you get: >>> a = [0, False, 1, True] >>> print a [0, False, 1, True] >>> for x in a: print x 0 False 1 True From fredrik@pythonware.com Wed Apr 3 18:18:50 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 3 Apr 2002 20:18:50 +0200 Subject: [Python-Dev] Re: PEP 285: Adding a bool type References: <016701c1db1a$569543e0$0900a8c0@spiff><00af01c1db2a$e7316d10$ced241d5@hagrid><200204031619.g33GJXm02577@europa.research.att.com><016b01c1db31$8da51010$ced241d5@hagrid> Message-ID: <031001c1db3c$0460ad90$ced241d5@hagrid> Andrew Koenig wrote: > On the other hand, the library isn't nearly so consistent. Just for > fun, I started browsing the library documentation at random. > > The very first page I visited was the description of gc, the > garbage collector interface. The first function that returns > a truth value there is described as > > isenabled() > Returns true if automatic collection is enabled. >>> import gc >>> gc.isenabled() 1 >>> type(gc.isenabled()) > endswith(suffix[,start[,end]]) > Return true if the string ends with the specified suffix, > otherwise return false. >>> "spam".endswith("m") 1 >>> type("spam".endswith("m")) > isatty() > Return true if the file is connected to a tty(-like) device, > else false. >>> sys.stdout.isatty() 1 >>> type(sys.stdout.isatty()) > exists(path) > Return true if path refers to an existing path. >>> os.path.exists("/etc/passwd") 1 >>> type(os.path.exists("/etc/passwd")) looks pretty consistent to me ;-) (Python docs tend to use "true" to mean anything which isn't "false", and where operator.truth returns the right thing. this won't change post-PEP -- if you start interpreting "return true" as "return True", you'll be asking for trouble) From marklists@mceahern.com Wed Apr 3 18:26:32 2002 From: marklists@mceahern.com (Mark McEahern) Date: Wed, 3 Apr 2002 12:26:32 -0600 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: Message-ID: [Ka-Ping Yee] > What about this: > > >>> d = {} > >>> d[0] = 'zero' > >>> d[False] = 'false' > >>> len(d) > 1 or 2? This returns: 1 Just curious: If you were using True and False as keys to a dictionary, would you ever expect len(d) to return something > 2? // mark From fdrake@acm.org Wed Apr 3 18:27:46 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 3 Apr 2002 13:27:46 -0500 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: References: <016701c1db1a$569543e0$0900a8c0@spiff> <00af01c1db2a$e7316d10$ced241d5@hagrid> <200204031619.g33GJXm02577@europa.research.att.com> <016b01c1db31$8da51010$ced241d5@hagrid> Message-ID: <15531.18850.911468.547377@grendel.zope.com> Andrew Koenig writes: > The very first page I visited was the description of gc, the > garbage collector interface. The first function that returns > a truth value there is described as > > isenabled() > Returns true if automatic collection is enabled. I spent a bit of time talking to Guido about the effect of PEP 285 on the documentation. My own preference is to do the same as the examples you cite: use the English words "true" and "false", and not give concrete typed values. Guido would rather *not* have it this way, but rather document the exact concrete return values. He'd have me refer to "True" and "False" in the same way we refer to "None", and refer to the return type as "bool" (if indeed we're using the new bool type), or, if 1 and 0 are the return values, document them specifically as well. For function arguments, he'd like to see truth values described as Boolean, and the values would be the English "true" and "false". Translations would be able to translate the English words, but not the True and False values, which are language constants (as constant as built-ins are). Guido & I may clash over this a few times. ;-( -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From fdrake@acm.org Wed Apr 3 18:29:55 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 3 Apr 2002 13:29:55 -0500 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: <031001c1db3c$0460ad90$ced241d5@hagrid> References: <016701c1db1a$569543e0$0900a8c0@spiff> <00af01c1db2a$e7316d10$ced241d5@hagrid> <200204031619.g33GJXm02577@europa.research.att.com> <016b01c1db31$8da51010$ced241d5@hagrid> <031001c1db3c$0460ad90$ced241d5@hagrid> Message-ID: <15531.18979.357453.21774@grendel.zope.com> Fredrik Lundh writes: > (Python docs tend to use "true" to mean anything which isn't > "false", and where operator.truth returns the right thing. this > won't change post-PEP -- if you start interpreting "return true" as > "return True", you'll be asking for trouble) That's right, but see my other note: Guido would like things like this to say "return True", which is different. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From jeremy@zope.com Wed Apr 3 18:36:51 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Wed, 3 Apr 2002 13:36:51 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format In-Reply-To: <15531.16299.945226.362456@slothrop.zope.com> References: <15531.10824.991521.687697@magrathea.basistech.com> <20020403165253.GA23796@ute.mems-exchange.org> <15531.16299.945226.362456@slothrop.zope.com> Message-ID: <15531.19395.258188.646213@slothrop.zope.com> David, I just posted a squishy comment on PEP 287. I realized that I can be more specific: PEP 287 describes a general-purpose structured text format "allowing for all kinds of documents." It doesn't say much of anything about docstrings specifically: "This PEP is not concerned with docstring semantics or processing at all." I think this is backwards. The PEP proposes to solve a specific problem -- docstring formats -- without any discussion of the problem domain or its requirements. Moreover, I disagree with goal 5. I think it is a non-goal to write a module's entire documentation in docstrings. (Perhaps it should be a goal that the docstring format is *not* right enough to write a module's entire documentation <0.6 wink>.) I think there two missing goals that are essential for a PEP that will make people happy: The first goal should be to keep the markup as simple as possible. The second goal should be to be targetted specifically at the needs of people who write docstrings for the standard library. I think it would also be productive to see an example or two of how this new format would be used in the standard library. Take a module that already has some decent docstrings and re-write it in the new format. Then we can see what benefit results from the effort. And take a module without docstrings and write new ones. (Some candidates: string, random, and unittest have good docstrings. getopt and hmac aren't bad. weakref is relatively small and doesn't have any docstrings.) If the primary goal is to keep the markup simple, I think it's impossible to judge a candidate without knowing what markup is required for docstrings. I am uncomfortable with the PEP's argument that "The problem is that a construct that seems superfluous to one person may be absolutely essential to another." A good design for a docstring format makes some hard decisions about what actually is essential and what is bloat. To paraphrase Aristole, wisdom comes from choosing wisely in the particular situation. Jeremy From guido@python.org Wed Apr 3 18:56:52 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 03 Apr 2002 13:56:52 -0500 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: Your message of "Wed, 03 Apr 2002 12:06:31 CST." References: Message-ID: <200204031856.g33Iuqr22751@odiug.zope.com> > It's the *speaking*. When i want to say true or false, then there's > the dilemma. > > I know, your answer is "you should always just say True or False", > and Mark McEahern said the same thing. But this cannot be so in > practice: *everything* already returns 0 or 1. (It may be possible to > get around this if we commit to changing the entire standard library > before releasing a version of Python with True and False, but alas, > this is not the only issue... read on.) > > As long as True and False are somewhere represented as 0 and 1, > the values 0 and 1 will never lose legitimacy as booleans. This > business with str() and/or repr() producing "0" or "1" for backwards > compatibility prevents us from considering 0 and 1 relegated to a > truly non-boolean status. But that's a variant of the PEP that no-one except Marc-Andre has spoken in favor of. The PEP proposes str(True) == "True". > Consider this: > > >>> a = [0, False, 1, True] > > >>> print a > [0, 0, 1, 1] > > >>> for x in a: print x > 0 > 0 > 1 > 1 > > Good heavens! It will print 0 False 1 True > What about this: > > >>> d = {} > >>> d[0] = 'zero' > >>> d[False] = 'false' > >>> len(d) > 1 or 2? What about this: >>> d = {} >>> d[0] = 'int' >>> d[0.0] = 'float' >>> d[0j] = 'complex' >>> print len(d) 1 or 3? False and True are numbers, and they are equal (==) to 0 and 1; everything else follows from there. > Basically, the concept of having a permanently schizophrenic type > in the language scares me. The above shows, i believe, that a > reasonable implementation must print True as True and False as False, > and never mention 1 or 0. And this is what the PEP proposes (despite its brief mention of an alternative). > Moreover, as soon as you start sorting a > bag of objects, or keying dictionaries on objects, you are forced to > run into the distinction between 0 and False, and between 1 and True. No you're not. d[0] and d[False] retrieve the same value, as do d[0L], d[0.0], and d[0j]. > I'm not against the idea of booleans, of course -- but i do think > that halfway booleans are worse than what we have now. And getting > to real booleans [*] involves real pain; it's just a question of > whether that pain is worth it. Even if we get all the way there -- > as in we manage to convert enough code and convince everyone to use > the new style -- i will never ever want "and" and "or" to return > booleans (i just hope that doesn't confuse anyone). > > > -- ?!ng > > [*] By real booleans, i mean the following. (Booleans would have > to behave like this for me to consider them "good enough" to > be better than what we have now.) > > >>> False, repr(False), str(False) > (False, 'False', 'False') > >>> True, repr(False), str(False) > (True, 'False', 'False') > >>> False + True > TypeError... That's just one textbook idea of what a Boolean "should" be. > >>> False == None > 0 > >>> False == 0 > 0 > >>> True == 1 > 0 > >>> {0: 0, False: False, 1: 1, True: True} > {0: 0, False: False, 1: 1, True: True} > > ... and probably > > >>> None < False < True < 0 > True > > (Hee hee -- i suppose the fact that "boolean" starts with a "b" > gets us this for free. But i wonder how many people are going > to be puzzled by True < 0?) Yuck. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Apr 3 18:58:38 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 03 Apr 2002 13:58:38 -0500 Subject: [Python-Dev] RE: Re: PEP 285: Adding a bool type In-Reply-To: Your message of "Wed, 03 Apr 2002 13:08:13 EST." References: Message-ID: <200204031858.g33IwcC22766@odiug.zope.com> > Having canonical True and False values would lead newcomers > to compare to them. This would work, for example: > > DONE = y and (x > y) > ... > if DONE is False: > ... > > On the other hand, this wouldn't (unless y is a boolean): > > DONE = (x > y) and y > ... > if DONE is False: > ... > > Of course, it works either way if you replace "DONE is False" > with "not DONE". Tutorial should point this out. It's poor coding style even in languages that do have strict Booleans, so I think it's a good idea to nip this one in the bud. > Or if you replace "y" with "bool(y)". What a waste to write "if bool(y) == True" instead of "if y" ... :-( --Guido van Rossum (home page: http://www.python.org/~guido/) From jafo-python-dev@tummy.com Wed Apr 3 19:01:28 2002 From: jafo-python-dev@tummy.com (Sean Reifschneider) Date: Wed, 3 Apr 2002 12:01:28 -0700 Subject: [Python-Dev] New canonical location for PEPs In-Reply-To: ; from martin@v.loewis.de on Wed, Apr 03, 2002 at 08:00:30PM +0200 References: <15530.31727.557290.92540@grendel.zope.com> <20020402233722.H16962@tummy.com> <20020403004949.I16962@tummy.com> Message-ID: <20020403120128.J16962@tummy.com> On Wed, Apr 03, 2002 at 08:00:30PM +0200, Martin v. Loewis wrote: >Yes, I know - while it solves this aspect of the problem, it isn't >really simple, either. I wouldn't call it overly complex... I'd be happy to help with getting it set up. >It may be that he has done something *like* you propose. However, I >still doubt that you could do *exactly* that. The important thing is the end result. I admitted that I really don't know the details of CVS use in this way, and that was the best I could find through 15 minutes of hunting on google and in the man page. I don't know the exact mechanism used in the case I cited, but I'll see if I can find it. Sean -- A ship in port is safe, but that is not what ships are for. -- Rear Admiral Grace Murray Hopper Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From jeremy@zope.com Wed Apr 3 18:58:36 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Wed, 3 Apr 2002 13:58:36 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format In-Reply-To: <15531.19395.258188.646213@slothrop.zope.com> References: <15531.10824.991521.687697@magrathea.basistech.com> <20020403165253.GA23796@ute.mems-exchange.org> <15531.16299.945226.362456@slothrop.zope.com> <15531.19395.258188.646213@slothrop.zope.com> Message-ID: <15531.20700.225267.683023@slothrop.zope.com> One more response. I just chatted with Guido, and he helped me see a different purpose for the PEP. It sounds like reStructuredText (reST) is intended for people who do want to write all the documentation in docstrings. If that's the goal, then it's fine if the doc-sig wants to settle on reST as the answer for those people. I wouldn't object to seeing this PEP approved as an informational PEP that described reST as an optional format for docstrings. (I'm assuming that there is consensus in the doc-sig that reST is the right solution.) As such, the PEP shouldn't be trying to convince people to use reST so much as it should describe the reST format. If the PEP is just going to be an advocacy document, there's not much point to a PEP. Or maybe the PEP could just say "reST is documented elsewhere. The doc-sig has agreed on this as the standard all-things-to-all-people format for the following reasons: ..." As an optional format, I think it would be helpful to explicitly note that it will not be used for the Python standard library. We've already got pretty good documentation for the library in LaTex, and I can't think of any reason to move all that text into the source code of the modules. Jeremy From tim.one@comcast.net Wed Apr 3 19:02:05 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 03 Apr 2002 14:02:05 -0500 Subject: [Python-Dev] pymalloc killer In-Reply-To: <464156E0633B154091731247D911C9CF0E0CF5@ge0057excuag01.ags.agere.com> Message-ID: >> #define ADDRESS_IN_RANGE(P, I) \ >> ((I) < narenas && (uptr)(P) - arenas[I] < (uptr)ARENA_SIZE) [Marangozov, Vladimir] > Bravo! Very nice. Surprisingly so, yes . > Now that the pool header is reduced in size by one slot, I can only > suggest a small optimization to get rid off the multiplication when > trying to extend the free list. Since you know the code by heart > already, this should be self explanatory. It's only 3 lines of code: > > 1) add a member in the pool struct, after capacity: > > uint lastoffset; /* free list tail block offset */ > > 2) In malloc, in the "try to extend the free list" block: > > size <<= ALIGNMENT_SHIFT; /* block size */ > pool->lastoffset += size; > pool->freeblock = (block *)pool + pool->lastoffset; > > 3) In the "init_pool" block: > > pool->capacity = ... > pool->lastoffset = POOL_OVERHEAD + size; > UNLOCK(); > > In other words, the lastoffset acts as an upper bound watermark. I added a bunch of low-level overviews (arean mgmt, pool mgmt, block mgmt) as comment blocks over the last week, and noted the possibility for this in one of them. Another possibility is to get rid of the runtime divisions, and the pool_header capacity member, by exploiting that pool->capacity = (POOL_SIZE - POOL_OVERHEAD) / size; is invariant across all pools with the same szidx; i.e., all references to pool->capacity *could* be replaced by capacity[pool->szidx], where "capacity" is a new small (at most 32 uints) file-static const vector computed at compile-time. > I didn't want to do that optimization before, because it would have > resulted in a bigger pool header and waste of space. Now it's ok. So now the tradeoff is a little muddier: If I skipped the multiplication optimization, but did the division optimization, each pool would suddenly gain 8 more bytes (on 32-bit boxes) to use for blocks. If I did both, the pool_header size would remain as it is now, "halfway between" multiple-of-8-byte sizes. How do you judge the tradeoffs now (do only one, or do both)? I don't foresee that anyone will have enough time to do the kinds of careful studies you did when first developing these algorithms, so happy to defer to your educated intuition on this. From guido@python.org Wed Apr 3 19:07:21 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 03 Apr 2002 14:07:21 -0500 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: Your message of "Wed, 03 Apr 2002 20:18:50 +0200." <031001c1db3c$0460ad90$ced241d5@hagrid> References: <016701c1db1a$569543e0$0900a8c0@spiff><00af01c1db2a$e7316d10$ced241d5@hagrid><200204031619.g33GJXm02577@europa.research.att.com><016b01c1db31$8da51010$ced241d5@hagrid> <031001c1db3c$0460ad90$ced241d5@hagrid> Message-ID: <200204031907.g33J7Lo22817@odiug.zope.com> > (Python docs tend to use "true" to mean anything which isn't > "false", and where operator.truth returns the right thing. this > won't change post-PEP -- if you start interpreting "return true" as > "return True", you'll be asking for trouble) Unclear. I intend to change as many functions as I reasonably can to return True/False, and I intend to change their documentation to match. While some of the documentation indeed talks of "true" and "false", those are not well-defined values. They are properties of values, and I find it bad style to say "f() returns true if X and false otherwise". I would have written that either as "f() returns a true value if X and a false value otherwise" (in case I don't want to commit to returning 0 or 1), or "f() returns 1 if X, 0 otherwise" (in case I do). --Guido van Rossum (home page: http://www.python.org/~guido/) From ark@research.att.com Wed Apr 3 19:20:19 2002 From: ark@research.att.com (Andrew Koenig) Date: 03 Apr 2002 14:20:19 -0500 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: <031001c1db3c$0460ad90$ced241d5@hagrid> References: <016701c1db1a$569543e0$0900a8c0@spiff> <00af01c1db2a$e7316d10$ced241d5@hagrid> <200204031619.g33GJXm02577@europa.research.att.com> <016b01c1db31$8da51010$ced241d5@hagrid> <031001c1db3c$0460ad90$ced241d5@hagrid> Message-ID: Fredrik> looks pretty consistent to me ;-) It may look that way, but if the documentation is to be believed, it's just happenstance. Fredrik> (Python docs tend to use "true" to mean anything which isn't Fredrik> "false", and where operator.truth returns the right thing. Fredrik> this won't change post-PEP -- if you start interpreting Fredrik> "return true" as "return True", you'll be asking for trouble) Indeed. In other words, the library does not have a consistent convention for what values to return for answers to yes/no questions. A particular implementation may happen to follow such a convention, but it is under no obligation to do so. -- Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark From pobrien@orbtech.com Wed Apr 3 19:25:05 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Wed, 3 Apr 2002 13:25:05 -0600 Subject: [Python-Dev] PEP 285: Adding a bool type In-Reply-To: <200203300539.g2U5dAR17743@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido van Rossum] > > I offer the following PEP for review by the community. If it receives > a favorable response, it will be implemented in Python 2.3. Yikes! I've read every single response to this so far. It has been quite enlightening for me. At the same time, I hope it ends soon. ;-) At this point I'm +1 on the proposal, but I have a favor to ask. While a lot of the issues raised have been revealing and provocative, I think the vast majority are tangential to the real substance of this PEP. I'm having trouble separating what is important from what is not. Is there any way we can focus the issue a bit? Specifically: It feels like there are too many combinations available in the PEP. (Or maybe too many questions and options which could result in weird combinations.) I'd like to see a new version with Guido's latest thoughts. For example, are we closer to a resolution on the str() and repr() issues? The only behavior I like is that exhibited by the sample code: >>> str(True) 'True' >>> repr(True) 'True' How about the case of the constants: true, false vs. True, False? I like the latter, which match None. They would also stand out in the new docstrings mentioned by Fred. How about operator.truth()? Has Tim changed his mind at all? Has Guido? If this PEP were implemented and "all built-in operations that conceptually return a Boolean" were updated, what verifiable examples of code breakage would we have? The issue of dictionary keys has been raised, and is a bit perplexing, but not overly so, IMO. However, I'd like confirmation that this is indeed the intended behavior: >>> d = {} >>> d[0] = 'First' >>> d {0: 'First'} >>> d[False] = 'Second' >>> d {0: 'Second'} >>> d[0.0] = 'Third' >>> d {0: 'Third'} >>> d[()] = 'Fourth' >>> d {0: 'Third', (): 'Fourth'} >>> d[None] = 'Fifth' >>> d {0: 'Third', (): 'Fourth', None: 'Fifth'} Or are there any situations like this where you would want True and False to be a bit more like None and a bit less like ints? Meaning True and False could be dictionary keys separate from 0 and 1. Other than that, are there any concrete issues that have been raised that need to be addressed by the PEP? I understand the uneasy feeling felt by many as I felt it myself. However, I've read all the newsgroup postings and reread the PEP and I don't feel very queasy any more. I think this PEP is a step in the right direction. I would hope that clarifying some of the genuine issues might help to bring about consensus and reduce some of the rhetoric. --- Patrick K. O'Brien Orbtech From pobrien@orbtech.com Wed Apr 3 19:34:37 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Wed, 3 Apr 2002 13:34:37 -0600 Subject: [Python-Dev] PEP 285: Adding a bool type In-Reply-To: Message-ID: > The issue of dictionary keys has been raised, and is a bit perplexing, but > not overly so, IMO. However, I'd like confirmation that this is indeed the > intended behavior: > > >>> d = {} > >>> d[0] = 'First' > >>> d > {0: 'First'} > >>> d[False] = 'Second' > >>> d > {0: 'Second'} > >>> d[0.0] = 'Third' > >>> d > {0: 'Third'} > >>> d[()] = 'Fourth' > >>> d > {0: 'Third', (): 'Fourth'} > >>> d[None] = 'Fifth' > >>> d > {0: 'Third', (): 'Fourth', None: 'Fifth'} > > Or are there any situations like this where you would want True > and False to > be a bit more like None and a bit less like ints? Meaning True and False > could be dictionary keys separate from 0 and 1. I see that you just addressed this and confirmed that True==1 and False==0 in all contexts. That's fine by me. I just wanted confirmation. --- Patrick K. O'Brien Orbtech From guido@python.org Wed Apr 3 19:34:40 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 03 Apr 2002 14:34:40 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format In-Reply-To: Your message of "Wed, 03 Apr 2002 13:36:51 EST." <15531.19395.258188.646213@slothrop.zope.com> References: <15531.10824.991521.687697@magrathea.basistech.com> <20020403165253.GA23796@ute.mems-exchange.org> <15531.16299.945226.362456@slothrop.zope.com> <15531.19395.258188.646213@slothrop.zope.com> Message-ID: <200204031934.g33JYfN22909@odiug.zope.com> [Jeremy] > To paraphrase Aristole, wisdom comes from choosing wisely in > the particular situation. I thought that wisdom came from experience, and experience from lack of wisdom. (Or is that common sense? There's a lot of wisdom in common sense.) I believe that reStructuredText comes from a lot of experience: long discussions about requirements in the doc-sig, and experience with e.g. Zope's StructuredText (which definitely represents the "lack of experience" position IMO). I think that reStructuredText is a good format for marking up docstrings; it's probably as good as it gets given the requirements (a fairly elaborate feature set, yet more readable "in the raw" than HTML). But if you ask me "should we use this for the standard library" I think I'll have to say no. Python's library reference documentation is written using LaTeX, like it or not, and it's not going to change any time soon. (*If* and *when* it changes, it's probably going to be something XMLish. But since XML is so more verbose than LaTeX, I'm not sure there's much of a point to this, at least unless bitrot takes the LaTeX toolchain away from us.) Given this status quo, docstrings in the Python standard library should not try to duplicate the library reference documentation; instead, they should be no more than concise hints. For such docstrings, a markup language, even reStructuredText, is inappropriate. IOW, reStructuredText is not even an option for new standard library modules. I agree with Jeremy that the PEP needs to be clear and explicit about this. --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@pythoncraft.com Wed Apr 3 19:36:42 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 3 Apr 2002 14:36:42 -0500 Subject: [Python-Dev] Cross-posting Message-ID: <20020403193642.GA2076@panix.com> Please be careful to avoid cross-posting between python-list and python-dev. I really hate reading messages twice. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ Why is this newsgroup different from all other newsgroups? From guido@python.org Wed Apr 3 19:46:55 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 03 Apr 2002 14:46:55 -0500 Subject: [Python-Dev] PEP 285: Adding a bool type In-Reply-To: Your message of "Wed, 03 Apr 2002 13:25:05 CST." References: Message-ID: <200204031946.g33JktP25857@odiug.zope.com> > It feels like there are too many combinations available in the > PEP. (Or maybe too many questions and options which could result in > weird combinations.) I'd like to see a new version with Guido's > latest thoughts. My thoughts haven't changed at all. :-) The first round of discussion (on python-dev) brought up some contentious issues where people disagreed strongly with particular details proposed in the PEP, and I put those contentious issues at the top of the review list. If you go to the latest version of the PEP on the web (http://www.python.org/peps/pep-0285.html) you'll see that I've added some other issues that were brought up in the more recent discussion. > For example, are we closer to a resolution on the > str() and repr() issues? The only behavior I like is that exhibited > by the sample code: > > >>> str(True) > 'True' > >>> repr(True) > 'True' So far I believe only Marc-Andre has expressed himself against this, so I'll make it so. > How about the case of the constants: true, false vs. True, False? I > like the latter, which match None. They would also stand out in the > new docstrings mentioned by Fred. Ditto. > How about operator.truth()? Has Tim changed his mind at all? Has Guido? I don't know about Tim. I've made an executive decision that operator.truth() will return a bool (making it an alternate spelling for bool()). > If this PEP were implemented and "all built-in operations that > conceptually return a Boolean" were updated, what verifiable > examples of code breakage would we have? I've changed the list of operations explicitly mentioned in the PEP (comparisons, some builtins, some string and file methods, a few other things). It breaks about 12 tests in the test suite. I haven't looked at all of these in detail, but I believe they are all shallow, of the following form: a test does "print x == y" and then the output is matched to "expected" output. The expected output strings must be changed from 0/1 to False/True. > The issue of dictionary keys has been raised, and is a bit > perplexing, but not overly so, IMO. However, I'd like confirmation > that this is indeed the intended behavior: > > >>> d = {} > >>> d[0] = 'First' > >>> d > {0: 'First'} > >>> d[False] = 'Second' > >>> d > {0: 'Second'} > >>> d[0.0] = 'Third' > >>> d > {0: 'Third'} > >>> d[()] = 'Fourth' > >>> d > {0: 'Third', (): 'Fourth'} > >>> d[None] = 'Fifth' > >>> d > {0: 'Third', (): 'Fourth', None: 'Fifth'} Yes. > Or are there any situations like this where you would want True and > False to be a bit more like None and a bit less like ints? Meaning > True and False could be dictionary keys separate from 0 and 1. No. That would break backwards compatibility. False==0, and True==1; everything else follows from that. (But False is not 0, and True is not 1!) > Other than that, are there any concrete issues that have been raised > that need to be addressed by the PEP? I understand the uneasy > feeling felt by many as I felt it myself. However, I've read all the > newsgroup postings and reread the PEP and I don't feel very queasy > any more. I think this PEP is a step in the right direction. I would > hope that clarifying some of the genuine issues might help to bring > about consensus and reduce some of the rhetoric. There's an educational issue that I'm just getting my head around. It appears that for various psychological and linguistic reasons, newbies naturally write if b == True: ... when they first encounter a bool variable and have to test whether it's true. I believe that this is most of Laura's issue. The only reasonable solution for this is that the Zen master slaps them over the head with a stick, and they learn to write if b: ... I'll update the PEP. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Apr 3 19:54:37 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 03 Apr 2002 14:54:37 -0500 Subject: [Python-Dev] Cross-posting In-Reply-To: Your message of "Wed, 03 Apr 2002 14:36:42 EST." <20020403193642.GA2076@panix.com> References: <20020403193642.GA2076@panix.com> Message-ID: <200204031954.g33Jsbo26224@odiug.zope.com> > Please be careful to avoid cross-posting between python-list and > python-dev. I really hate reading messages twice. Too bad. --Guido van Rossum (home page: http://www.python.org/~guido/) From gsw@agere.com Wed Apr 3 20:02:02 2002 From: gsw@agere.com (Gerald S. Williams) Date: Wed, 3 Apr 2002 15:02:02 -0500 Subject: [Python-Dev] RE: Re: PEP 285: Adding a bool type In-Reply-To: <200204031858.g33IwcC22766@odiug.zope.com> Message-ID: Guido van Rossum wrote: > Tutorial should point this out. It's poor coding style even in > languages that do have strict Booleans, so I think it's a good idea to > nip this one in the bud. Agreed, although one of Python's big selling points (for me, at least) is that you generally don't have to refer to the manual to understand what's happening, even if you're new to the language. And something like this may be a bit more subtle, since it is somewhat an issue of style. I don't expect people to go to this extreme: if (a > b) is True: ... But that would be valid code. There's probably *somebody* in the world who would even prefer the useless "is True". > What a waste to write "if bool(y) == True" instead of "if y" ... :-( True again. I think we all agree that comparisons to True and False would cause problems. I guess the question is how often people would actually get tripped up by it. Or worse, *not* get tripped up by it and start doing it regularly. :-) I was thinking you could simply attack the problem by doing something with the == and != operators, but that doesn't help if you test using "is" or "is not". I'm not sure if managing people's expectations ever really works, but perhaps that's the only real answer (assuming the PEP is implemented). -Jerry -O Gerald S. Williams, 55A-134A-E : mailto:gsw@agere.com O- -O AGERE SYSTEMS, 6755 SNOWDRIFT RD : office:610-712-8661 O- -O ALLENTOWN, PA, USA 18106-9353 : mobile:908-672-7592 O- From barry@zope.com Wed Apr 3 20:05:13 2002 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 3 Apr 2002 15:05:13 -0500 Subject: [Python-Dev] Re: PEP 285: Adding a bool type References: <200204031619.g33GJXm02577@europa.research.att.com> Message-ID: <15531.24697.971828.59708@anthem.wooz.org> >>>>> "KY" == Ka-Ping Yee writes: KY> This doesn't mean anything in practice. Every yes/no function KY> i've encountered returns 1 for yes and 0 for no. Now we see a way out. Rename the constants to Yes and No and add a builtin alias for bool() calld theAnswerTo(). Thus all our conditionals would be written like so: if theAnswerTo(myQuestion) is Yes: # do it truthful-ly y'rs, -Barry From pobrien@orbtech.com Wed Apr 3 20:14:44 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Wed, 3 Apr 2002 14:14:44 -0600 Subject: [Python-Dev] PEP 285: Adding a bool type In-Reply-To: <200204031946.g33JktP25857@odiug.zope.com> Message-ID: [Guido van Rossum] > My thoughts haven't changed at all. :-) > > The first round of discussion (on python-dev) brought up some > contentious issues where people disagreed strongly with particular > details proposed in the PEP, and I put those contentious issues at the > top of the review list. If you go to the latest version of the PEP on > the web (http://www.python.org/peps/pep-0285.html) you'll see that > I've added some other issues that were brought up in the more recent > discussion. I apologize for not having read the web version. I wasn't aware that it was being updated during the debate. I have a feeling I'm not alone in this mistake, so I'll take this opportunity to encourage others to read the web version also. > There's an educational issue that I'm just getting my head around. It > appears that for various psychological and linguistic reasons, newbies > naturally write > > if b == True: ... > > when they first encounter a bool variable and have to test whether > it's true. I believe that this is most of Laura's issue. The only > reasonable solution for this is that the Zen master slaps them over > the head with a stick, and they learn to write > > if b: ... Yes. And I think describing Python's handling of truth values as "something vs. nothing" has real merit as a learning aid. It will certainly keep me from falling into the if b == True: trap. In summary, I agree with your clarifications and I'm +1 on the PEP. --- Patrick K. O'Brien Orbtech From tim.one@comcast.net Wed Apr 3 20:27:13 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 03 Apr 2002 15:27:13 -0500 Subject: [Python-Dev] PEP 285: Adding a bool type In-Reply-To: <200204031946.g33JktP25857@odiug.zope.com> Message-ID: [pobrien@orbtech.com] > How about operator.truth()? Has Tim changed his mind at all? Has Guido? Guido change his mind? That's about as likely as Tim changing his . Guido Pronounced on this and that's good enough for me. I'd still prefer operator.truth() return 0/1, but this always was (as the PEP said from the start) a "minor additional issue". People can learn to use int(bool(x)) when they need 0/1, and operator.truth() becomes dead weight. From David Abrahams" <200204031856.g33Iuqr22751@odiug.zope.com> Message-ID: <024301c1db59$99b801f0$1101a8c0@boostconsulting.com> From: "Guido van Rossum" > False and True are numbers, and they are equal (==) to 0 and 1; > everything else follows from there. FWIW, that's the way it is in C++, and almost all of the same conclusions fall out of it. Also, as distasteful as it may seem intellectually, I have yet to hear any complaints about this situation in practice... so it seems to work out OK. I also note that this is not something people seem to have trouble with in making the transition from C to C++, which is analogous to the problem of legacy 1/0 code. -Dave From jeremy@zope.com Wed Apr 3 21:53:55 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Wed, 3 Apr 2002 16:53:55 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format In-Reply-To: <200204031934.g33JYfN22909@odiug.zope.com> References: <15531.10824.991521.687697@magrathea.basistech.com> <20020403165253.GA23796@ute.mems-exchange.org> <15531.16299.945226.362456@slothrop.zope.com> <15531.19395.258188.646213@slothrop.zope.com> <200204031934.g33JYfN22909@odiug.zope.com> Message-ID: <15531.31219.887534.212983@slothrop.zope.com> >>>>> "GvR" == Guido van Rossum writes: GvR> [Jeremy] >> To paraphrase Aristole, wisdom comes from choosing wisely in the >> particular situation. GvR> I thought that wisdom came from experience, and experience from GvR> lack of wisdom. (Or is that common sense? There's a lot of GvR> wisdom in common sense.) Oh, yes, I remember that one, too :-). Don't think it was Aristotle, but the saying goes: Good decisions come from experience; experience comes from bad decisions. GvR> I believe that reStructuredText comes from a lot of experience: GvR> long discussions about requirements in the doc-sig, and GvR> experience with e.g. Zope's StructuredText (which definitely GvR> represents the "lack of experience" position IMO). I believe that reST is a good structured text design, and I think it would be good to use it, e.g., to replace Zope's StructuredText. Jeremy From tim.one@comcast.net Wed Apr 3 22:09:07 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 03 Apr 2002 17:09:07 -0500 Subject: [Python-Dev] Suggested memory API rules for 2.3 In-Reply-To: <15530.57231.202946.180190@12-248-41-177.client.attbi.com> Message-ID: > Use as Function Designators > > Programs shall not use use a name from line B, C, D, F, G or H > in a C context requiring a function designator. Names from lines > A and E may be used in C contexts requiring a function designator. > A function designator is a C expression having function type; > C contexts requiring a function designator include as an actual > argument to a function taking a function argument. [Skip Montanaro] > This just means that PyMem_{Malloc, Realloc, Free} and PyObject_{Malloc, > Realloc, Free} will be implemented as functions or simple macros that > expand to a single function call, and that the others can be all sorts > of CPP gobbledygook, right? That was my intent, but it's hard to predict how courts will rule . > The only thing that threw me a little was the alphabet soup Fred needs to work with me to make the real docs comprehensible; all this stuff is on top of what's already said in the docs, and I believe rearrangement is called for (like combining the scattered sections about the memory APIs, for a start). From tim.one@comcast.net Wed Apr 3 22:24:02 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 03 Apr 2002 17:24:02 -0500 Subject: [Python-Dev] Re: Suggested memory API rules for 2.3 In-Reply-To: <20020403071619.A369@glacier.arctrix.com> Message-ID: [Neil Schemenauer] > Very good. Spelling out the rules was long overdue. That still needs to be done for PyObject_GC_xyz. "Shall not mix", "shall hold the GIL", "guaranteed binary compatibility", and "shall not use in contexts requiring function designators" are the no-brainer answers. Any objections there? >> Note that the eight ways to spell "free" all have to map to the >> pymalloc free when pymalloc is enabled in 2.3. There is no way to >> spell "give me the raw platform free(), damn it", except for "free". > I guess we need to do something special for pymalloc free() if > PYMALLOC_DEBUG is defined. I expect the simplest is to redirect all the "free" spellings to _PyMalloc_DebugFree then; then I'll have to change that function to pass the address on to system free() or _PyMalloc_Free(), depending on where the address came from. An alternative with some real attractions is to redirect all "get memory" spellings to _PyMalloc_DebugMalloc in PYMALLOC_DEBUG mode, in which case I'd need to change _PyMalloc_DebugMalloc to serialize via a lock, and change _PyMalloc_DebugFree to serialize too and just *verify* that all addresses passed to it came from _PyMalloc_Debug{Malloc, Realloc}. >> Recommended Practice >> ... > This should probably come up front in the documentation. All the rest > is historical crap. :-) I agree, but this is Fred's call in the end. In some ways the memory API docs need to be more precise, but in others they need to give succinct friendly advice about the least-hassle way to proceed. Alas, the compatability maze is so large it's bound to dominate the docs no matter how it's rearranged. From tim.one@comcast.net Wed Apr 3 22:24:02 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 03 Apr 2002 17:24:02 -0500 Subject: [Python-Dev] Re: Suggested memory API rules for 2.3 In-Reply-To: <2mn0wkena2.fsf@starship.python.net> Message-ID: [the recommended 2.3+ API] > PyMem_{Malloc, Realloc, Free} > PyObject_{Malloc, Realloc, Free} > PyObject_{New, NewVar} [Michael Hudson] > We should probably provide an example of preprocessor hackery that > lets you use these interfaces and remain backward compatible as far as > possible. Maybe even a little .h file which we can suggest extension > authors bundle with their extensions. I would be delighted if somebody else did that. As I age, I find it increasingly difficult to do more than 103 things at once . From aahz@pythoncraft.com Wed Apr 3 22:31:43 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 3 Apr 2002 17:31:43 -0500 (EST) Subject: [Python-Dev] Yes and no (was Re: PEP 285: Adding a bool type) References: <3CAA5777.66E7C2D5@python.org> Message-ID: <200204032231.g33MVhM08064@panix1.panix.com> In article <3CAA5777.66E7C2D5@python.org>, Guido van Rossum wrote: > >I'm reading (at least skimming) everything, but the more I read the >reactions, the more I believe that this is just the response I have to >expect for *every* change to the language I propose, and the more I'm >tempted to follow my gut instincts. Long ago (right after lambda :-) >I learned to say "no" to proposals that didn't feel right to me, even >if I couldn't express very clearly why. I think it's time I learn to >say "yes" even if the community doesn't see the light yet. I note that you've certainly done that in the past. ;-) One critical difference between "yes" and "no" in this context is that a "no" can be retracted, but "yes" rarely can. So I think that a bias toward "no" is overall good. But yes, we have to trust your instincts in the end; that's why we're all using Python in the first place. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ Why is this newsgroup different from all other newsgroups? -- From guido@python.org Wed Apr 3 22:32:52 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 03 Apr 2002 17:32:52 -0500 Subject: [Python-Dev] ACCEPTED: PEP 285 Message-ID: <200204032232.g33MWqJ28933@odiug.zope.com> Here's one final posting of PEP 285. I've read (or at least skimmed) all feedback that was on our news server by 9:30am EST on April 2nd, plus anything mailed directly to me or to python-dev until now. There was a lot of negative feedback, but there was also a lot of support for the PEP. The feedback made it easy to decide on most of the smaller issues in the review section of the PEP as previously posted: even reviewers who were against the PEP mostly supported my preferences for the various details. Despite the negative feedback, I've decided to accept the PEP. The most important perceived problem is that newbies tend to write if x == True: ... where they should write if x: ... I believe this problem should be solved by education (the Zen master hits the student on the head with a stick, and the student gets enlightenment) rather than by holding back what I believe will be a useful feature. You can expect the bool type in Python 2.3; it will be in CVS tonight. --Guido van Rossum (home page: http://www.python.org/~guido/) PEP: 285 Title: Adding a bool type Version: $Revision: 1.20 $ Last-Modified: $Date: 2002/04/03 22:11:05 $ Author: guido@python.org (Guido van Rossum) Status: Accepted Type: Standards Track Created: 8-Mar-2002 Python-Version: 2.3 Post-History: 8-Mar-2002, 30-Mar-2002, 3-Apr-2002 Abstract This PEP proposes the introduction of a new built-in type, bool, with two constants, False and True. The bool type would be a straightforward subtype (in C) of the int type, and the values False and True would behave like 0 and 1 in most respects (for example, False==0 and True==1 would be true) except repr() and str(). All built-in operations that conceptually return a Boolean result will be changed to return False or True instead of 0 or 1; for example, comparisons, the "not" operator, and predicates like isinstance(). Review I've collected enough feedback to last me a lifetime, so I declare the review period officially OVER. I had Chinese food today; my fortune cookie said "Strong and bitter words indicate a weak cause." It reminded me of some of the posts against this PEP... :-) Anyway, here are my BDFL pronouncements. (Executive summary: I'm not changing a thing; all variants are rejected.) 1) Should this PEP be accepted? => Yes. There have been many arguments against the PEP. Many of them were based on misunderstandings. I've tried to clarify some of the most common misunderstandings below in the main text of the PEP. The only issue that weighs at all for me is the tendency of newbies to write "if x == True" where "if x" would suffice. More about that below too. I think this is not a sufficient reason to reject the PEP. 2) Should str(True) return "True" or "1"? "1" might reduce backwards compatibility problems, but looks strange. (repr(True) would always return "True".) => "True". Almost all reviewers agree with this. 3) Should the constants be called 'True' and 'False' (similar to None) or 'true' and 'false' (as in C++, Java and C99)? => True and False. Most reviewers agree that consistency within Python is more important than consistency with other languages. 4) Should we strive to eliminate non-Boolean operations on bools in the future, through suitable warnings, so that for example True+1 would eventually (in Python 3000) be illegal? => No. There's a small but vocal minority that would prefer to see "textbook" bools that don't support arithmetic operations at all, but most reviewers agree with me that bools should always allow arithmetic operations. 5) Should operator.truth(x) return an int or a bool? => bool. Tim Peters believes it should return an int, but almost all other reviewers agree that it should return a bool. My rationale: operator.truth() exists to force a Boolean context on its argument (it calls the C API PyObject_IsTrue()). Whether the outcome is reported as int or bool is secondary; if bool exists there's no reason not to use it. (Under the PEP, operator.truth() now becomes an alias for bool(); that's fine.) 6) Should bool inherit from int? => Yes. In an ideal world, bool might be better implemented as a separate integer type that knows how to perform mixed-mode arithmetic. However, inheriting bool from int eases the implementation enormously (in part since all C code that calls PyInt_Check() will continue to work -- this returns true for subclasses of int). Also, I believe this is right in terms of substitutability: code that requires an int can be fed a bool and it will behave the same as 0 or 1. Code that requires a bool may not work when it is given an int; for example, 3 & 4 is 0, but both 3 and 4 are true when considered as truth values. 7) Should the name 'bool' be changed? => No. Some reviewers have argued for boolean instead of bool, because this would be easier to understand (novices may have heard of Boolean algebra but may not make the connection with bool) or because they hate abbreviations. My take: Python uses abbreviations judiciously (like 'def', 'int', 'dict') and I don't think these are a burden to understanding. To a newbie, it doesn't matter whether it's called a waffle or a bool; it's a new word, and they learn quickly what it means. One reviewer has argued to make the name 'truth'. I find this an unattractive name, and would actually prefer to reserve this term (in documentation) for the more abstract concept of truth values that already exists in Python. For example: "when a container is interpreted as a truth value, an empty container is considered false and a non-empty one is considered true." 8) Should we strive to require that Boolean operations (like "if", "and", "not") have a bool as an argument in the future, so that for example "if []:" would become illegal and would have to be writen as "if bool([]):" ??? => No!!! Some people believe that this is how a language with a textbook Boolean type should behave. Because it was brought up, others have worried that I might agree with this position. Let me make my position on this quite clear. This is not part of the PEP's motivation and I don't intend to make this change. (See also the section "Clarification" below.) Rationale Most languages eventually grow a Boolean type; even C99 (the new and improved C standard, not yet widely adopted) has one. Many programmers apparently feel the need for a Boolean type; most Python documentation contains a bit of an apology for the absence of a Boolean type. I've seen lots of modules that defined constants "False=0" and "True=1" (or similar) at the top and used those. The problem with this is that everybody does it differently. For example, should you use "FALSE", "false", "False", "F" or even "f"? And should false be the value zero or None, or perhaps a truth value of a different type that will print as "true" or "false"? Adding a standard bool type to the language resolves those issues. Some external libraries (like databases and RPC packages) need to be able to distinguish between Boolean and integral values, and while it's usually possible to craft a solution, it would be easier if the language offered a standard Boolean type. This also applies to Jython: some Java classes have separately overloaded methods or constructors for int and boolean arguments. The bool type can be used to select the boolean variant. (The same is apparently the case for some COM interfaces.) The standard bool type can also serve as a way to force a value to be interpreted as a Boolean, which can be used to normalize Boolean values. When a Boolean value needs to be normalized to one of two values, bool(x) is much clearer than "not not x" and much more concise than if x: return 1 else: return 0 Here are some arguments derived from teaching Python. When showing people comparison operators etc. in the interactive shell, I think this is a bit ugly: >>> a = 13 >>> b = 12 >>> a > b 1 >>> If this was: >>> a > b True >>> it would require a millisecond less thinking each time a 0 or 1 was printed. There's also the issue (which I've seen baffling even experienced Pythonistas who had been away from the language for a while) that if you see: >>> cmp(a, b) 1 >>> cmp(a, a) 0 >>> you might be tempted to believe that cmp() also returned a truth value, whereas in reality it can return three different values (-1, 0, 1). If ints were not (normally) used to represent Booleans results, this would stand out much more clearly as something completely different. Specification The following Python code specifies most of the properties of the new type: class bool(int): def __new__(cls, val=0): # This constructor always returns an existing instance if val: return True else: return False def __repr__(self): if self: return "True" else: return "False" __str__ = __repr__ def __and__(self, other): if isinstance(other, bool): return bool(int(self) & int(other)) else: return int.__and__(self, other) __rand__ = __and__ def __or__(self, other): if isinstance(other, bool): return bool(int(self) | int(other)) else: return int.__or__(self, other) __ror__ = __or__ def __xor__(self, other): if isinstance(other, bool): return bool(int(self) ^ int(other)) else: return int.__xor__(self, other) __rxor__ = __xor__ # Bootstrap truth values through sheer willpower False = int.__new__(bool, 0) True = int.__new__(bool, 1) The values False and True will be singletons, like None. Because the type has two values, perhaps these should be called "doubletons"? The real implementation will not allow other instances of bool to be created. True and False will properly round-trip through pickling and marshalling; for example pickle.loads(pickle.dumps(True)) will return True, and so will marshal.loads(marshal.dumps(True)). All built-in operations that are defined to return a Boolean result will be changed to return False or True instead of 0 or 1. In particular, this affects comparisons (<, <=, ==, !=, >, >=, is, is not, in, not in), the unary operator 'not', the built-in functions callable(), hasattr(), isinstance() and issubclass(), the dict method has_key(), the string and unicode methods endswith(), isalnum(), isalpha(), isdigit(), islower(), isspace(), istitle(), isupper(), and startswith(), the unicode methods isdecimal() and isnumeric(), and the 'closed' attribute of file objects. The predicates in the operator module are also changed to return a bool, including operator.truth(). Because bool inherits from int, True+1 is valid and equals 2, and so on. This is important for backwards compatibility: because comparisons and so on currently return integer values, there's no way of telling what uses existing applications make of these values. It is expected that over time, the standard library will be updated to use False and True when appropriate (but not to require a bool argument type where previous an int was allowed). This change should not pose additional problems and is not specified in detail by this PEP. C API The header file "boolobject.h" defines the C API for the bool type. It is included by "Python.h" so there is no need to include it directly. The existing names Py_False and Py_True reference the unique bool objects False and True (previously these referenced static int objects with values 0 and 1, which were not unique amongst int values). A new API, PyObject *PyBool_FromLong(long), takes a C long int argument and returns a new reference to either Py_False (when the argument is zero) or Py_True (when it is nonzero). To check whether an object is a bool, the macro PyBool_Check() can be used. The type of bool instances is PyBoolObject *. The bool type object is available as PyBool_Type. Clarification This PEP does *not* change the fact that almost all object types can be used as truth values. For example, when used in an if statement, an empty list is false and a non-empty one is true; this does not change and there is no plan to ever change this. The only thing that changes is the preferred values to represent truth values when returned or assigned explicitly. Previously, these preferred truth values were 0 and 1; the PEP changes the preferred values to False and True, and changes built-in operations to return these preferred values. Compatibility Because of backwards compatibility, the bool type lacks many properties that some would like to see. For example, arithmetic operations with one or two bool arguments is allowed, treating False as 0 and True as 1. Also, a bool may be used as a sequence index. I don't see this as a problem, and I don't want evolve the language in this direction either. I don't believe that a stricter interpretation of "Booleanness" makes the language any clearer. Another consequence of the compatibility requirement is that the expression "True and 6" has the value 6, and similarly the expression "False or None" has the value None. The "and" and "or" operators are usefully defined to return the first argument that determines the outcome, and this won't change; in particular, they don't force the outcome to be a bool. Of course, if both arguments are bools, the outcome is always a bool. It can also easily be coerced into being a bool by writing for example "bool(x and y)". Resolved Issues (See also the Review section above.) - Because the repr() or str() of a bool value is different from an int value, some code (for example doctest-based unit tests, and possibly database code that relies on things like "%s" % truth) may fail. It is easy to work around this (without explicitly referencing the bool type), and it is expected that this only affects a very small amount of code that can easily be fixed. - Other languages (C99, C++, Java) name the constants "false" and "true", in all lowercase. For Python, I prefer to stick with the example set by the existing built-in constants, which all use CapitalizedWords: None, Ellipsis, NotImplemented (as well as all built-in exceptions). Python's built-in namespace uses all lowercase for functions and types only. - It has been suggested that, in order to satisfy user expectations, for every x that is considered true in a Boolean context, the expression x == True should be true, and likewise if x is considered false, x == False should be true. In particular newbies who have only just learned about Boolean variables are likely to write if x == True: ... instead of the correct form, if x: ... There seem to be strong psychological and linguistic reasons why many people are at first uncomfortable with the latter form, but I believe that the solution should be in education rather than in crippling the language. After all, == is general seen as a transitive operator, meaning that from a==b and b==c we can deduce a==c. But if any comparison to True were to report equality when the other operand was a true value of any type, atrocities like 6==True==7 would hold true, from which one could infer the falsehood 6==7. That's unacceptable. (In addition, it would break backwards compatibility. But even if it didn't, I'd still be against this, for the stated reasons.) Newbies should also be reminded that there's never a reason to write if bool(x): ... since the bool is implicit in the "if". Explicit is *not* better than implicit here, since the added verbiage impairs redability and there's no other interpretation possible. There is, however, sometimes a reason to write b = bool(x) This is useful when it is unattractive to keep a reference to an arbitrary object x, or when normalization is required for some other reason. It is also sometimes appropriate to write i = int(bool(x)) which converts the bool to an int with the value 0 or 1. This conveys the intention to henceforth use the value as an int. Implementation A complete implementation in C has been uploaded to the SourceForge patch manager: http://python.org/sf/528022 This will soon be checked into CVS for python 2.3a0. Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil fill-column: 70 End: From nas@python.ca Wed Apr 3 22:48:53 2002 From: nas@python.ca (Neil Schemenauer) Date: Wed, 3 Apr 2002 14:48:53 -0800 Subject: [Python-Dev] Re: Suggested memory API rules for 2.3 In-Reply-To: ; from tim.one@comcast.net on Wed, Apr 03, 2002 at 05:24:02PM -0500 References: <20020403071619.A369@glacier.arctrix.com> Message-ID: <20020403144853.A1509@glacier.arctrix.com> Tim Peters wrote: > That still needs to be done for PyObject_GC_xyz. "Shall not mix", "shall > hold the GIL", "guaranteed binary compatibility", Yes. > "shall not use in contexts requiring function designators" People need some form of GC free to initialize tp_free. Extensions can't really use _PyObject_GC_Del, can they? Neil From DavidA@ActiveState.com Wed Apr 3 22:47:08 2002 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 03 Apr 2002 14:47:08 -0800 Subject: [Python-Dev] ACCEPTED: PEP 285 References: <200204032232.g33MWqJ28933@odiug.zope.com> Message-ID: <3CAB866C.4DF51F72@activestate.com> Guido van Rossum wrote: > Despite the negative feedback, I've decided to accept the PEP. The > most important perceived problem is that newbies tend to write > > if x == True: ... > > where they should write > > if x: ... > > I believe this problem should be solved by education (the Zen master > hits the student on the head with a stick, and the student gets > enlightenment) rather than by holding back what I believe will be a > useful feature. Would it not be "relatively" easy to add a compiler-time warning for most uses of this dangerous idiom? At the very least, the boolean type could do the equivalent of: def __eq__(self, other): if __debug__ and other not in (0,1,True,False): warnings.warn("Testing of equality is best done by asking objects whether they're true, not comparing with a boolean", RunTimeWarning) return int.__eq__(self, other) or something similar? --david From ping@lfw.org Wed Apr 3 22:53:55 2002 From: ping@lfw.org (Ka-Ping Yee) Date: Wed, 3 Apr 2002 16:53:55 -0600 (CST) Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: <200204031856.g33Iuqr22751@odiug.zope.com> Message-ID: Guido van Rossum wrote: > False and True are numbers [...] That's exactly where this falls down. Patrick O'Brien wrote: > [...] True and False could be dictionary keys separate from 0 and 1. Guido van Rossum wrote: > No. That would break backwards compatibility. False==0, and True==1; > everything else follows from that. (But False is not 0, and True is > not 1!) This is a strange distinction to make -- one that is not made in any other programming language i have ever encountered. Of course *i* get it and *you* get it -- but it will be so hard to teach this particular weirdness that i consider it fatal to the proposal. -- ?!ng From fdrake@acm.org Wed Apr 3 22:54:42 2002 From: fdrake@acm.org (Fred L. Drake) Date: Wed, 3 Apr 2002 17:54:42 -0500 (EST) Subject: [Python-Dev] [development doc updates] Message-ID: <20020403225442.0343418EAD1@grendel.zope.com> The development version of the documentation has been updated: http://www.python.org/dev/doc/devel/ Updated to include whatever Guido did about bool(). From tim.one@comcast.net Wed Apr 3 22:59:46 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 03 Apr 2002 17:59:46 -0500 Subject: [Python-Dev] Suggested memory API rules for 2.3 In-Reply-To: <200204031657.g33GuuH22110@odiug.zope.com> Message-ID: >> "Programs" does not include the Python core. We can do anything we >> need to do in the core, based on our complete knowledge of >> each release's internal implementation details. [Guido] > However, since many people use the core as an example, we may have to > be careful here. Certainly xxmodule.c, xxobject.c and xxsubtype.c > should do the recommended thing; I'll make sure that they do. > but if too many of the built-in objects use the macros despite those > being in danger of deprecation, that might still perpetuate use of > those macros. It's hard to imagine that the macro names will often be cheaper than the function names, given the new requirement that they have identical I/O behavior. So, yes, I expect a lot of mindless source edits to replace core use of macro names with function names (and I'm good at massive mindless source edits ). There may be still be cases where the core wants to exploit that, e.g., it *knows* a malloc argument is > 0, and so skip the _PyMem_EXTRA business; if so, I expect we'll grow an internal API spelling for that. >> After memory has been obtained via one of these functions, it >> should be resized and freed only by a function from the same line, >> except that PyMem_Free may be used freely in place of PyMem_Del, >> and PyObject_Free in place of PyObject_Del. > I actually had a slightly different rule in mind: I did too at first. > mixing and matching between two adjacent lines that only differ by case > is also permitted. Except of course that the macro lines will be > deprecated, so maybe it doesn't matter to spell this out. Exactly so. Since programs should not use macro names at all anymore, text devoted to drawing fine distinctions among macro name use is pretty much a waste. Note there is no "shall" or "shall not" text *forbidding* mixing corresponding macro and function names: the text above is only "should be ... from the same line". The later blurbs of the form Memory obtained by PyMem_{Malloc, MALLOC} shall be resized only by PyMem_{Realloc, REALLOC}. are *not* intended not to be read as forbidding mixing PyMem_Malloc with PyMem_REALLOC. Perhaps that would be clearer said Memory obtained by either of PyMem_{Malloc, MALLOC} shall be resized only by either of PyMem_{Realloc, REALLOC}. The exhaustive tables I posted earlier are much clearer for stuff like this, but, again, I'm decreasingly keen to spell out elaborate details for spellings that are discouraged. [on whether private-API spellings for raw malloc etc should be introduced] > IMO, PyMem_Malloc() and PyMem_Free() should be good enough. I was expecting this would be an objective matter resolved by exhaustive study of the code base when renaming the macro uses. I'm happy to settle it arbitrarily, though . >> Relationship to Platform Allocator >> >> All names in lines A, B, C and D ultimately invoke the platform >> C malloc, realloc, or free. However, programs shall not mix any >> of these names with direct calls to the platform malloc, calloc, >> realloc, or free referencing the same base memory addresses, as >> Python may need to perform bookkeeping of its own around its calls >> to the platform allocator. > Really? Why not just say these are wrappers around malloc and free? I'm specifically trying to leave the door open for the PyMalloc_DebugXYZ routines to capture all uses of memory API functions in PYMALLOC_DEBUG mode. It also reads better as-is than to say, well, ya, OK, I suppose you *can* mix raw calls to libc with these particular memory API families, but the PyObject_GC_xyz family is entirely different in this respect. I don't want to draw distinctions just because the current implementation makes it possible to draw them; users shall consider malloc/realloc/free as a distinct API. > (On platforms where it matters, they will be guaranteed to use the > same heap as the core uses -- this is apparently an issue with Windows > DLLs.) But that's another reason (if it's real ) to say "no mixing" here: if this is a use they care about, then, e.g., they're dead meat if they mix PyMem_Malloc() with raw free() (the former would use the malloc in the Python DLL, the latter the free in the user's DLL). From David Abrahams" Message-ID: <031c01c1db63$15870520$1101a8c0@boostconsulting.com> ----- Original Message ----- From: "Ka-Ping Yee" > Guido van Rossum wrote: > > No. That would break backwards compatibility. False==0, and True==1; > > everything else follows from that. (But False is not 0, and True is > > not 1!) > > This is a strange distinction to make -- one that is not made in any > other programming language i have ever encountered. Of course *i* get > it and *you* get it -- but it will be so hard to teach this particular > weirdness that i consider it fatal to the proposal. C++ is /exactly/ like that. I think also C99, but don't quote me. From jeremy@zope.com Wed Apr 3 23:02:57 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Wed, 3 Apr 2002 18:02:57 -0500 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: <031c01c1db63$15870520$1101a8c0@boostconsulting.com> References: <031c01c1db63$15870520$1101a8c0@boostconsulting.com> Message-ID: <15531.35361.331408.203697@slothrop.zope.com> >>>>> "DA" == David Abrahams writes: DA> C++ is /exactly/ like that. I think also C99, but don't quote DA> me. I believe C99 defines true to be 1 and false to be 0. Jeremy From guido@python.org Wed Apr 3 23:05:49 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 03 Apr 2002 18:05:49 -0500 Subject: [Python-Dev] ACCEPTED: PEP 285 In-Reply-To: Your message of "Wed, 03 Apr 2002 14:47:08 PST." <3CAB866C.4DF51F72@activestate.com> References: <200204032232.g33MWqJ28933@odiug.zope.com> <3CAB866C.4DF51F72@activestate.com> Message-ID: <200204032305.g33N5nm29961@odiug.zope.com> > > Despite the negative feedback, I've decided to accept the PEP. The > > most important perceived problem is that newbies tend to write > > > > if x == True: ... > > > > where they should write > > > > if x: ... > > > > I believe this problem should be solved by education (the Zen master > > hits the student on the head with a stick, and the student gets > > enlightenment) rather than by holding back what I believe will be a > > useful feature. > > Would it not be "relatively" easy to add a compiler-time warning for > most uses of this dangerous idiom? I think this would be a job for PyChecker. Neal? --Guido van Rossum (home page: http://www.python.org/~guido/) From neal@metaslash.com Wed Apr 3 23:13:29 2002 From: neal@metaslash.com (Neal Norwitz) Date: Wed, 03 Apr 2002 18:13:29 -0500 Subject: [Python-Dev] ACCEPTED: PEP 285 References: <200204032232.g33MWqJ28933@odiug.zope.com> Message-ID: <3CAB8C99.232A8571@metaslash.com> Guido van Rossum wrote: > > Despite the negative feedback, I've decided to accept the PEP. The > most important perceived problem is that newbies tend to write > > if x == True: ... > > where they should write > > if x: ... > > I believe this problem should be solved by education (the Zen master > hits the student on the head with a stick, and the student gets > enlightenment) rather than by holding back what I believe will be a > useful feature. You can expect the bool type in Python 2.3; it will > be in CVS tonight. And the warnings have already been added to pychecker. I know I'm going to wish I never said this, but ... If you find REAL problems with code that breaks, let me know. I will try to address problems that may arise. This code: True = 1 False = 0 def test(a): if a is True: print 'True' if a == False: print 'False' generates these warnings: test.py:2: Should not assign to True, it is (or will be) a builtin test.py:3: Should not assign to False, it is (or will be) a builtin test.py:6: Comparisions with True are not necessary test.py:7: Comparisions with False are not necessary Neal From pobrien@orbtech.com Wed Apr 3 23:22:25 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Wed, 3 Apr 2002 17:22:25 -0600 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: Message-ID: > Guido van Rossum wrote: > > False and True are numbers [...] > > That's exactly where this falls down. > > Patrick O'Brien wrote: > > [...] True and False could be dictionary keys separate from 0 and 1. > > Guido van Rossum wrote: > > No. That would break backwards compatibility. False==0, and True==1; > > everything else follows from that. (But False is not 0, and True is > > not 1!) > > Ka-Ping Yee wrote: > This is a strange distinction to make -- one that is not made in any > other programming language i have ever encountered. Of course *i* get > it and *you* get it -- but it will be so hard to teach this particular > weirdness that i consider it fatal to the proposal. This strange distinction seems to be the crux of the controversy surrounding this PEP. I thought it a bit strange at first too. But the more I think about it the less it troubles me. In fact, it is starting to feel downright Pythonic. True and False are singleton instances of a Boolean subclass of the Integer class. As a subclass, much of, but not all of, their behavior is inherited from Integer. So False often acts like 0 and True often acts like 1. But not always. Because they are instances of a subclass. That's what subclassing allows. This is Object Orientation 101. I think the concepts truly are no less difficult to understand or explain than plenty of other aspects of the Python language. The way Boolean is being implemented may not be what one would expect, and it may not be what other languages do, and it may take a little getting used to, and it may upset some people's preconceptions of truth values, but I think if everyone steps back and looks at this objectively they have to admit that this "strange distinction" really isn't too terribly strange after all. There are plenty of Python features that still make my head hurt. This new Boolean isn't one of them. --- Patrick K. O'Brien Orbtech From python@rcn.com Wed Apr 3 23:26:17 2002 From: python@rcn.com (Raymond Hettinger) Date: Wed, 3 Apr 2002 18:26:17 -0500 Subject: [Python-Dev] Dict.popitem(key=None) Message-ID: <002c01c1db66$f4c54280$3ee87ad1@othello> This is a multi-part message in MIME format. ------=_NextPart_000_0029_01C1DB3D.0B3698E0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Dan Parisien posted a small feature request about three months ago: = http://sourceforge.net/tracker/index.php?func=3Ddetail&aid=3D495086&group= _id=3D5470&atid=3D355470 The current behavior of popitem is to return and remove an arbitrary = key/value pair from a dictionary. With the optional argument, Dan = proposes that a specific key/value pair be returned and removed. = Without the optional argument, the behavior would be unchanged. I think this is a great idea. Like .get() and .setdefault(), this = provides a fast, explicit alternative to a commonly used series of = dictionary operations. The speed improvement comes from doing a single = lookup for retrieving the item and then deleting it. Like the optional = argument in list.pop(i), the extra ability is added without cluttering = the default case. If we can agree that this should be done and accepted, I'll volunteer to = write the patch. Raymond Hettinger Before -------- value =3D adict[key] del key return (key, value) After ------ return adict.popitem(key) ------=_NextPart_000_0029_01C1DB3D.0B3698E0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
 
The current behavior of popitem is to = return and=20 remove an arbitrary key/value pair from a dictionary.  With = the=20 optional argument, Dan proposes that a specific key/value pair be = returned and=20 removed.  Without the optional argument, the behavior would be=20 unchanged.
 
I think this is a great idea.  = Like .get() and=20 .setdefault(), this provides a fast, explicit alternative to a commonly = used=20 series of dictionary operations.  The speed improvement comes from = doing a=20 single lookup for retrieving the item and then deleting it.  = Like the optional argument in list.pop(i), the = extra ability=20 is added without cluttering the default case.
 
If we can agree that this should be = done and=20 accepted, I'll volunteer to write the patch.
 
 
Raymond Hettinger
 
 
 
Before
--------
value =3D adict[key]
del key
return (key, value)
 
After
------
return adict.popitem(key)
 
------=_NextPart_000_0029_01C1DB3D.0B3698E0-- From DavidA@ActiveState.com Wed Apr 3 23:37:55 2002 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 03 Apr 2002 15:37:55 -0800 Subject: [Python-Dev] ACCEPTED: PEP 285 References: <200204032232.g33MWqJ28933@odiug.zope.com> <3CAB866C.4DF51F72@activestate.com> <200204032305.g33N5nm29961@odiug.zope.com> Message-ID: <3CAB9253.F5FFD2F7@activestate.com> Guido van Rossum wrote: > > > > Despite the negative feedback, I've decided to accept the PEP. The > > > most important perceived problem is that newbies tend to write > > > > > > if x == True: ... > > > > > > where they should write > > > > > > if x: ... > > > > > > I believe this problem should be solved by education (the Zen master > > > hits the student on the head with a stick, and the student gets > > > enlightenment) rather than by holding back what I believe will be a > > > useful feature. > > > > Would it not be "relatively" easy to add a compiler-time warning for > > most uses of this dangerous idiom? > > I think this would be a job for PyChecker. Neal? Neal just posted that PyChecker deals with it, which is great, but I don't think that PyChecker is enough - until PyChecker is part of the standard distribution and integrated with e.g. the interactive shell, I think that PyChecker is not likely to be used by novices. And we're talking about novices here, right? Come to think of it, getting PyChecker integrated w/ the interactive shell would be great! Hard, I suspect, but great! --david From tim.one@comcast.net Wed Apr 3 23:47:39 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 03 Apr 2002 18:47:39 -0500 Subject: [Python-Dev] Re: Suggested memory API rules for 2.3 In-Reply-To: <20020403144853.A1509@glacier.arctrix.com> Message-ID: [Tim] >> That still needs to be done for PyObject_GC_xyz. "Shall not >> mix", "shall hold the GIL", "guaranteed binary compatibility", [Neil, the PyObject_GC Master] > Yes. >> "shall not use in contexts requiring function designators" > People need some form of GC free to initialize tp_free. Extensions > can't really use _PyObject_GC_Del, can they? No they can't. So we promise that PyObject_GC_Del can be used in contexts requiring function designators. We're sure that PyObject_GC_{New, NewVar, Resize} can't, since they cast their results to a type "passed" as an argument. Harmonic convergence? From neal@metaslash.com Thu Apr 4 00:00:07 2002 From: neal@metaslash.com (Neal Norwitz) Date: Wed, 03 Apr 2002 19:00:07 -0500 Subject: [Python-Dev] ACCEPTED: PEP 285 References: <200204032232.g33MWqJ28933@odiug.zope.com> <3CAB866C.4DF51F72@activestate.com> <200204032305.g33N5nm29961@odiug.zope.com> <3CAB9253.F5FFD2F7@activestate.com> Message-ID: <3CAB9787.A29ACB46@metaslash.com> David Ascher wrote: > > Guido van Rossum wrote: > > > > > > Despite the negative feedback, I've decided to accept the PEP. The > > > > most important perceived problem is that newbies tend to write > > > > > > > > if x == True: ... > > > > > > > > where they should write > > > > > > > > if x: ... > > > > > > > > I believe this problem should be solved by education (the Zen master > > > > hits the student on the head with a stick, and the student gets > > > > enlightenment) rather than by holding back what I believe will be a > > > > useful feature. > > > > > > Would it not be "relatively" easy to add a compiler-time warning for > > > most uses of this dangerous idiom? > > > > I think this would be a job for PyChecker. Neal? > > Neal just posted that PyChecker deals with it, which is great, but I > don't think that PyChecker is enough - until PyChecker is part of the > standard distribution and integrated with e.g. the interactive shell, I > think that PyChecker is not likely to be used by novices. And we're > talking about novices here, right? > > Come to think of it, getting PyChecker integrated w/ the interactive > shell would be great! Hard, I suspect, but great! Not too hard: >>> from pychecker import checker >>> import test_bool test_bool.py:6: Comparisions with True are not necessary test_bool.py:7: Comparisions with False are not necessary If you always wanted this, you could add a PYTHONSTARTUP script to do the import or you could add/modify sitecustomize.py. It probably isn't perfect since I don't use this, but it could work. Neal From DavidA@ActiveState.com Thu Apr 4 00:14:46 2002 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 03 Apr 2002 16:14:46 -0800 Subject: [Python-Dev] ACCEPTED: PEP 285 References: <200204032232.g33MWqJ28933@odiug.zope.com> <3CAB866C.4DF51F72@activestate.com> <200204032305.g33N5nm29961@odiug.zope.com> <3CAB9253.F5FFD2F7@activestate.com> <3CAB9787.A29ACB46@metaslash.com> Message-ID: <3CAB9AF6.5E543320@activestate.com> Neal Norwitz wrote: > > David Ascher wrote: > > Come to think of it, getting PyChecker integrated w/ the interactive > > shell would be great! Hard, I suspect, but great! > > Not too hard: > > >>> from pychecker import checker > >>> import test_bool > test_bool.py:6: Comparisions with True are not necessary > test_bool.py:7: Comparisions with False are not necessary > > If you always wanted this, you could add a PYTHONSTARTUP script > to do the import or you could add/modify sitecustomize.py. That's nice, but it doesn't deal with the real novice's usage, which is to learn at the >>> prompt, not in modules. E:\tmp>cat > foo.py cat: Reading input from keyboard: def foo(): a = 3 return 3 ^Z E:\tmp>python ActivePython 2.2.0 Build 221 (ActiveState Corp.) based on Python 2.2 (#28, Mar 28 2002, 12:10:20) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> from pychecker import checker >>> import foo foo.py:2: Local variable (a) not used >>> that's nice, but: >>> def foo(): ... a = 3 ... return 3 ... >>> The fact that it doesn't pick up on the unused variable is too bad. The integration I'm thinking of would be deeper. --da From tree@basistech.com Thu Apr 4 00:18:40 2002 From: tree@basistech.com (Tom Emerson) Date: Wed, 3 Apr 2002 19:18:40 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format In-Reply-To: <15531.31219.887534.212983@slothrop.zope.com> References: <15531.10824.991521.687697@magrathea.basistech.com> <20020403165253.GA23796@ute.mems-exchange.org> <15531.16299.945226.362456@slothrop.zope.com> <15531.19395.258188.646213@slothrop.zope.com> <200204031934.g33JYfN22909@odiug.zope.com> <15531.31219.887534.212983@slothrop.zope.com> Message-ID: <15531.39904.185110.261762@magrathea.basistech.com> Jeremy Hylton writes: > I believe that reST is a good structured text design, and I think it > would be good to use it, e.g., to replace Zope's StructuredText. Why does the community need a new structured text design? What is wrong with existing markup methodologies? The PEP didn't answer these with any cogent examples, IMHO. The only rationale that I saw was existing use: the people that already make use of it find it natural, and everyone else has to learn it. -- Tom Emerson Basis Technology Corp. Sr. Computational Linguist http://www.basistech.com "Beware the lollipop of mediocrity: lick it once and you suck forever" From skip@pobox.com Thu Apr 4 00:33:53 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 3 Apr 2002 18:33:53 -0600 Subject: [Python-Dev] big performance hit in the past few days Message-ID: <15531.40817.280442.421426@12-248-41-177.client.attbi.com> After Guido checked in the bool() stuff I cvs up'd and rebuilt. A few days ago I spent some time trying to quantify the effect of changes to SMALL_REQUEST_THRESHOLD on pymalloc performance. The "benchmark" consists of using the compiler package to compile Lib/*.py and three runs of the pystone main program with LOOPS set to 100000 (10x the usual value). On March 31, I got the following output with and without pymalloc enabled and a SMALL_REQUEST_THRESHOLD of 256: w/ pymalloc Pystone(1.1) time for 100000 passes = 16.81 This machine benchmarks at 5948.84 pystones/second Pystone(1.1) time for 100000 passes = 16.82 This machine benchmarks at 5945.3 pystones/second Pystone(1.1) time for 100000 passes = 16.83 This machine benchmarks at 5941.77 pystones/second 243.84user 0.23system 4:12.73elapsed 96%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (402major+4432minor)pagefaults 0swaps w/o pymalloc Pystone(1.1) time for 100000 passes = 17.66 This machine benchmarks at 5662.51 pystones/second Pystone(1.1) time for 100000 passes = 17.67 This machine benchmarks at 5659.31 pystones/second Pystone(1.1) time for 100000 passes = 17.66 This machine benchmarks at 5662.51 pystones/second 277.88user 0.21system 4:48.10elapsed 96%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (400major+3943minor)pagefaults 0swaps Running the same benchmark just now I got: w/ pymalloc Pystone(1.1) time for 100000 passes = 25.1 This machine benchmarks at 3984.06 pystones/second Pystone(1.1) time for 100000 passes = 24.99 This machine benchmarks at 4001.6 pystones/second Pystone(1.1) time for 100000 passes = 24.74 This machine benchmarks at 4042.04 pystones/second 352.33user 0.97system 6:51.40elapsed 85%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (499major+4181minor)pagefaults 0swaps w/o pymalloc Pystone(1.1) time for 100000 passes = 25.01 This machine benchmarks at 3998.4 pystones/second Pystone(1.1) time for 100000 passes = 25.09 This machine benchmarks at 3985.65 pystones/second Pystone(1.1) time for 100000 passes = 25.18 This machine benchmarks at 3971.41 pystones/second 374.38user 0.26system 6:37.71elapsed 94%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (401major+3950minor)pagefaults 0swaps All files were compiled using gcc 3.0.4 with OPT set at -O3. The fact that the tests slowed down dramatically both with and without pymalloc enabled suggests that recent changes to obmalloc are not to blame. (On March 31, I was using obmalloc.c 2.24. Today I'm using 2.27.) Any thoughts on the possible cause? It's tough to casually suggest a particular culprit because the bool() stuff touched a lot of files. I can't simply identify a few files that changed in the past few days. I count 66 .c[ch] files new or updated since mid-afternoon April 1. Skip From guido@python.org Thu Apr 4 00:52:15 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 03 Apr 2002 19:52:15 -0500 Subject: [Python-Dev] ACCEPTED: PEP 285 In-Reply-To: Your message of "Wed, 03 Apr 2002 18:13:29 EST." <3CAB8C99.232A8571@metaslash.com> References: <200204032232.g33MWqJ28933@odiug.zope.com> <3CAB8C99.232A8571@metaslash.com> Message-ID: <200204040052.g340qGL07447@pcp742651pcs.reston01.va.comcast.net> > And the warnings have already been added to pychecker. I knew I could count on you. :-) > I know I'm going to wish I never said this, but ... > If you find REAL problems with code that breaks, let me know. > I will try to address problems that may arise. > > This code: > True = 1 > False = 0 I wonder if you should add warnings for *any* spelling, or at least also for true and TRUE, if assigned the constant 1 (and similar for false/FALSE), on the basis that these are probably other ways to spell the same thing. E.g. I've seen FALSE, TRUE = range(2) > def test(a): > if a is True: print 'True' But 'is True' can be a useful test. The newbies we're trying to catch in verboten activities are using ==. > if a == False: print 'False' > > generates these warnings: > test.py:2: Should not assign to True, it is (or will be) a builtin > test.py:3: Should not assign to False, it is (or will be) a builtin > test.py:6: Comparisions with True are not necessary > test.py:7: Comparisions with False are not necessary Cool. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Apr 4 00:55:03 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 03 Apr 2002 19:55:03 -0500 Subject: [Python-Dev] Dict.popitem(key=None) In-Reply-To: Your message of "Wed, 03 Apr 2002 18:26:17 EST." <002c01c1db66$f4c54280$3ee87ad1@othello> References: <002c01c1db66$f4c54280$3ee87ad1@othello> Message-ID: <200204040055.g340t3b07468@pcp742651pcs.reston01.va.comcast.net> > Dan Parisien posted a small feature request about three months ago: > http://sourceforge.net/tracker/index.php?func=detail&aid=495086&group_id=5470&atid=355470 > > The current behavior of popitem is to return and remove an arbitrary > key/value pair from a dictionary. With the optional argument, Dan > proposes that a specific key/value pair be returned and removed. > Without the optional argument, the behavior would be unchanged. > > I think this is a great idea. Like .get() and .setdefault(), this > provides a fast, explicit alternative to a commonly used series of > dictionary operations. The speed improvement comes from doing a > single lookup for retrieving the item and then deleting it. Like > the optional argument in list.pop(i), the extra ability is added > without cluttering the default case. > > If we can agree that this should be done and accepted, I'll > volunteer to write the patch. Without having thought too much about it, I'm inclined to agree. Go ahead and submit the patch to SF. If the populace doesn't want it, though, I will deny any responsibility. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Apr 4 00:59:09 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 03 Apr 2002 19:59:09 -0500 Subject: [Python-Dev] ACCEPTED: PEP 285 In-Reply-To: Your message of "Wed, 03 Apr 2002 15:37:55 PST." <3CAB9253.F5FFD2F7@activestate.com> References: <200204032232.g33MWqJ28933@odiug.zope.com> <3CAB866C.4DF51F72@activestate.com> <200204032305.g33N5nm29961@odiug.zope.com> <3CAB9253.F5FFD2F7@activestate.com> Message-ID: <200204040059.g340x9b08176@pcp742651pcs.reston01.va.comcast.net> > Neal just posted that PyChecker deals with it, which is great, but I > don't think that PyChecker is enough - until PyChecker is part of > the standard distribution and integrated with e.g. the interactive > shell, I think that PyChecker is not likely to be used by novices. > And we're talking about novices here, right? Yeah, but I think we should only complain about comparisons to bool that use the literals (as opposed to "if x == y" where either x or y happens to be a bool). A run-time warning can't really do that. > Come to think of it, getting PyChecker integrated w/ the interactive > shell would be great! Hard, I suspect, but great! Certainly better than running tabnanny. :-) I'm not even sure how hard it would be to add this to IDLE -- pychecker is already a module. Maybe Neal will enjoy this challenge? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Apr 4 01:01:53 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 03 Apr 2002 20:01:53 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format In-Reply-To: Your message of "Wed, 03 Apr 2002 19:18:40 EST." <15531.39904.185110.261762@magrathea.basistech.com> References: <15531.10824.991521.687697@magrathea.basistech.com> <20020403165253.GA23796@ute.mems-exchange.org> <15531.16299.945226.362456@slothrop.zope.com> <15531.19395.258188.646213@slothrop.zope.com> <200204031934.g33JYfN22909@odiug.zope.com> <15531.31219.887534.212983@slothrop.zope.com> <15531.39904.185110.261762@magrathea.basistech.com> Message-ID: <200204040101.g3411rB08206@pcp742651pcs.reston01.va.comcast.net> > Why does the community need a new structured text design? What is > wrong with existing markup methodologies? The PEP didn't answer these > with any cogent examples, IMHO. Maybe that's because it's obvious to anyone who has hung out on doc-sig long enough. Structured text is really a great idea for certain situations; reST is a much better implementation of the idea than any versions I've seen before. E.g. the ST code in Zope stinks (this is not a secret). > The only rationale that I saw was existing use: the people that > already make use of it find it natural, and everyone else has to learn > it. Maybe David needs to write up the rationale better. But I can assure you there is one. --Guido van Rossum (home page: http://www.python.org/~guido/) From stephen@xemacs.org Thu Apr 4 01:10:16 2002 From: stephen@xemacs.org (Stephen J. Turnbull) Date: 04 Apr 2002 10:10:16 +0900 Subject: [Python-Dev] New canonical location for PEPs In-Reply-To: <20020403004949.I16962@tummy.com> References: <15530.31727.557290.92540@grendel.zope.com> <20020402233722.H16962@tummy.com> <20020403004949.I16962@tummy.com> Message-ID: <87ofh0xo93.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Sean" == Sean Reifschneider writes: Sean> There is some reasonable mechanism for [automatically Sean> pushing content from CVS to the Web], as it *IS* done in Sean> some places. I know XEmacs's is not perfect, but I'm fairly sure host config changes at SourceForge and SunSITE.de have been responsible for most of the attention the webmaster has paid to the auto-update mechanism over the last two years. Attention has been paid to the lock issue, but I don't recall the resolution. We currently keep the website content in CVS at SunSITE.de. A CVS commit to the website module triggers updates of the SourceForge, Tux, and SunSITE "primary" homepages, typically taking 10-15 seconds to propagate to all three sites, including a "genpage" run. Adrian Aichner is the guy to talk to about the website experience (eg, admin resources consumed since original setup). I believe Martin Buchholz designed the mechanism and did the original implementation. Karl Hegbloom adapted it for a project he works on, and was the one who brought up the lock issue most recently. You may be able to browse the implementation in CVSROOT at cvs.xemacs.org. If not, I'm sure Adrian would be willing to show it to you. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Don't ask how you can "do" free software business; ask what your business can "do for" free software. From neal@metaslash.com Thu Apr 4 01:10:56 2002 From: neal@metaslash.com (Neal Norwitz) Date: Wed, 03 Apr 2002 20:10:56 -0500 Subject: [Python-Dev] ACCEPTED: PEP 285 References: <200204032232.g33MWqJ28933@odiug.zope.com> <3CAB866C.4DF51F72@activestate.com> <200204032305.g33N5nm29961@odiug.zope.com> <3CAB9253.F5FFD2F7@activestate.com> <200204040059.g340x9b08176@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CABA820.735A9958@metaslash.com> Guido van Rossum wrote: > > > Come to think of it, getting PyChecker integrated w/ the interactive > > shell would be great! Hard, I suspect, but great! > > Certainly better than running tabnanny. :-) > > I'm not even sure how hard it would be to add this to IDLE -- > pychecker is already a module. Maybe Neal will enjoy this challenge? Enjoyed, past-tense. :-) ls -l tmp/idle-patch -rw-rw-r-- 1 neal neal 1758 Aug 5 2001 tmp/idle-patch I'll update the patch to make it current and post on SF. Neal From guido@python.org Thu Apr 4 01:29:15 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 03 Apr 2002 20:29:15 -0500 Subject: [Python-Dev] ACCEPTED: PEP 285 In-Reply-To: Your message of "Wed, 03 Apr 2002 20:10:56 EST." <3CABA820.735A9958@metaslash.com> References: <200204032232.g33MWqJ28933@odiug.zope.com> <3CAB866C.4DF51F72@activestate.com> <200204032305.g33N5nm29961@odiug.zope.com> <3CAB9253.F5FFD2F7@activestate.com> <200204040059.g340x9b08176@pcp742651pcs.reston01.va.comcast.net> <3CABA820.735A9958@metaslash.com> Message-ID: <200204040129.g341TFJ09380@pcp742651pcs.reston01.va.comcast.net> > > I'm not even sure how hard it would be to add this to IDLE -- > > pychecker is already a module. Maybe Neal will enjoy this challenge? > > Enjoyed, past-tense. :-) Hey! Who told you you could have the keys to the time machine! --Guido van Rossum (home page: http://www.python.org/~guido/) From tree@basistech.com Thu Apr 4 02:15:55 2002 From: tree@basistech.com (Tom Emerson) Date: Wed, 3 Apr 2002 21:15:55 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format In-Reply-To: <200204040101.g3411rB08206@pcp742651pcs.reston01.va.comcast.net> References: <15531.10824.991521.687697@magrathea.basistech.com> <20020403165253.GA23796@ute.mems-exchange.org> <15531.16299.945226.362456@slothrop.zope.com> <15531.19395.258188.646213@slothrop.zope.com> <200204031934.g33JYfN22909@odiug.zope.com> <15531.31219.887534.212983@slothrop.zope.com> <15531.39904.185110.261762@magrathea.basistech.com> <200204040101.g3411rB08206@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15531.46939.994236.194689@magrathea.basistech.com> Guido van Rossum writes: > > Why does the community need a new structured text design? What is > > wrong with existing markup methodologies? The PEP didn't answer these > > with any cogent examples, IMHO. > > Maybe that's because it's obvious to anyone who has hung out on > doc-sig long enough. So in order to understand a PEP one needs to subscribe to all applicable mailing lists? > Structured text is really a great idea for certain situations; reST > is a much better implementation of the idea than any versions I've > seen before. E.g. the ST code in Zope stinks (this is not a > secret). I have no idea what is better. I ju > > The only rationale that I saw was existing use: the people that > > already make use of it find it natural, and everyone else has to learn > > it. > > Maybe David needs to write up the rationale better. But I can assure > you there is one. Obviously so, else he wouldn't have written the PEP in the first place. But the rationale in the PEP didn't convince me that it was worth my time to blindly adopt a new markup scheme that I've never used to document classes in the Python libraries and application I'm writing merely to make use of the documentation tools that are provided. Instead the greater motivation is to adopt JavaDoc/Doxygen and write appropriate tools, since most developers already speak enough HTML to write the reference documentation. Of course there is a lot of current practice which has the momentum. Given how carefully rationalized the other PEPs are, there is no reason to not make 287 equally rationalized. Devil's-advocately-yours, tree -- Tom Emerson Basis Technology Corp. Sr. Computational Linguist http://www.basistech.com "Beware the lollipop of mediocrity: lick it once and you suck forever" From fdrake@acm.org Thu Apr 4 02:19:18 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 3 Apr 2002 21:19:18 -0500 Subject: [Python-Dev] New canonical location for PEPs In-Reply-To: <87ofh0xo93.fsf@tleepslib.sk.tsukuba.ac.jp> References: <15530.31727.557290.92540@grendel.zope.com> <20020402233722.H16962@tummy.com> <20020403004949.I16962@tummy.com> <87ofh0xo93.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <15531.47142.871230.351524@grendel.zope.com> Stephen J. Turnbull writes: > I know XEmacs's is not perfect, but I'm fairly sure host config Close enough for me! ;-) > Adrian Aichner is the guy to talk to about the > website experience (eg, admin resources consumed since original > setup). I believe Martin Buchholz designed the > mechanism and did the original implementation. Karl Hegbloom > adapted it for a project he works on, and was the > one who brought up the lock issue most recently. You may be able to > browse the implementation in CVSROOT at cvs.xemacs.org. If not, I'm > sure Adrian would be willing to show it to you. It doesn't look like the CVSROOT directory is visible in your ViewCVS installation. I'd be interested in seeing how you weave the pieces together. The material on python.org is currently woven together from multiple CVS repositories (I can think of three, at least!). -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From tim.one@comcast.net Thu Apr 4 02:45:20 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 03 Apr 2002 21:45:20 -0500 Subject: [Python-Dev] Dict.popitem(key=None) In-Reply-To: <002c01c1db66$f4c54280$3ee87ad1@othello> Message-ID: [Raymond Hettinger, championing an optional arg to dict.popitem()] > If we can agree that this should be done and accepted, I'll volunteer > to write the patch. +1 on doing the patch, but I'm not so easily tricked into voting on code, docs and test cases that don't yet exist Message-ID: [Skip Montanaro, on a recent gross slowdown] > ... > Any thoughts on the possible cause? Nope. Barry hasn't checked anything in lately . > It's tough to casually suggest a particular culprit because the bool() > stuff touched a lot of files. I can't simply identify a few files that > changed in the past few days. I count 66 .c[ch] files new or updated > since mid-afternoon April 1. Do you know about the -D option to cvs update? It's a kind of time machine; you can update to the code as it existed at any specified time (like, say, one minute before mid-afternoon April 1). Alas, unlike Guido's time machine, it's sticky. I'm not sure I'm seeing a slowdown on Windows (it's *really* hard to time stuff on my Win98 home box), so maybe that's a clue too (but quite possibly a bogus clue). From bsder@allcaps.org Thu Apr 4 03:12:03 2002 From: bsder@allcaps.org (Andrew P. Lentvorski) Date: Wed, 3 Apr 2002 19:12:03 -0800 (PST) Subject: [Python-Dev] ACCEPTED: PEP 285 In-Reply-To: <200204032232.g33MWqJ28933@odiug.zope.com> Message-ID: <20020403175400.U14863-100000@mail.allcaps.org> I love the idea of a bool type, but please don't inherit bool from int. I didn't want to chime in before because I thought Ka-Ping Yee covered what a "real boolean" is better than I could, but his email didn't elicit much comment. The main disadvantage to a bool inheriting from an int is that there is no way that the Python interpreter can check to make sure that it really gets a bool if it inherits from int. The only way to check if you got a real bool is if the end programmer writes something using isinstance(). For compatibility, having True and False equate to 1 and 0, respectively, is a good idea. But please do it by promotion or conversion or something, not inheritance. Once the assumption of bool-is-an-int gets locked into the language, it will be impossible to switch to bool-only-looks-like-an-int or bool-is-a-real-type later. If bool is a separate type, Python can have a --anal-bool (name specifically chosen to be memorable ;) option which means "By gosh, if it isn't a bool and it should be, throw something." If bool inherits from int, I see no way to ever implement that check in the language. Please correct me if this is an erroneous assumption. As a concrete example, I have a VLSI layout tool written completely in Python. Because I and my colleagues are disciples of the Cult of Paranoid Programming(tm), the codebase of 7674 lines includes 31 lines (counting the else and complaint as one line) with checks of the following form: >>> def checkme(shouldBeBoolean): ... if shouldBeBoolean == 0: ... print "False here" ... elif shouldBeBoolean == 1: ... print "True here" ... else: ... print "Evil person!!! Don't hand me a non-boolean" ... >>> checkme(0) False here >>> checkme(1) True here >>> checkme([]) Evil person!!! Don't hand me a non-boolean >>> checkme("anything") Evil person!!! Don't hand me a non-boolean That means that about .5-1% (depending on how you really count the checking lines) of the code exists solely to check to make sure that a bool really is a bool. And it's drudgery, boilerplate code that adds no functionality to the program. This practice got adopted because we were running down too many bugs which started from [] or None sliding through as false and then causing subtle problems later in the program when it could have been caught and fixed immediately. However, to catch these bugs requires a programmer to write the extra check. With --anal-bool: the above sequence becomes: >>> def checkme(shouldBeBoolean): ... if shouldBeBoolean: ... print "True here" ... else: ... print "False here" With --anal-bool: >>> checkme(False) False here >>> checkme(True) True here >>> checkme(0) Exception: ... >>> checkme(1) Exception: ... >>> checkme([]) Exception: ... >>> checkme("anything") Exception: ... Without --anal-bool: >>> checkme(False) False here >>> checkme(True) True here >>> checkme(0) False here >>> checkme(1) True here >>> checkme([]) False Here >>> checkme("anything") True here By leaving the default as loose, the backward compatibility is preserved while still allowing stricter semantics. By having --anal-bool as a flag, I can remove .5-1% of my total codebase. And we all know that the only bug-free code is code that doesn't exist. ;) The --anal-bool flag will also help point out all those places in the Python code itself which will have to be changed to True and False. And do so loudly. ;) -a From stephen@xemacs.org Thu Apr 4 03:20:07 2002 From: stephen@xemacs.org (Stephen J. Turnbull) Date: 04 Apr 2002 12:20:07 +0900 Subject: [Python-Dev] ACCEPTED: PEP 285 In-Reply-To: <3CAB8C99.232A8571@metaslash.com> References: <200204032232.g33MWqJ28933@odiug.zope.com> <3CAB8C99.232A8571@metaslash.com> Message-ID: <87k7roxi8o.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Neal" == Neal Norwitz writes: Neal> This code [...] generates these warnings: Neal> test.py:6: Comparisions with True are not necessary That doesn't look like "shinai"[1] to me. Is it strong enough? Footnotes: [1] The split-bamboo sword preferred by Zen masters because it hurts more, with less permanent damage---to the bamboo, anyway. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Don't ask how you can "do" free software business; ask what your business can "do for" free software. From skip@pobox.com Thu Apr 4 03:55:53 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 3 Apr 2002 21:55:53 -0600 Subject: [Python-Dev] big performance hit in the past few days In-Reply-To: References: <15531.40817.280442.421426@12-248-41-177.client.attbi.com> Message-ID: <15531.52937.927063.8457@12-248-41-177.client.attbi.com> Tim> Do you know about the -D option to cvs update? Nope, thanks for the tip. I'm back up the time machine a bit and see what happens. Skip From ark@research.att.com Thu Apr 4 04:08:12 2002 From: ark@research.att.com (Andrew Koenig) Date: 03 Apr 2002 23:08:12 -0500 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: <024301c1db59$99b801f0$1101a8c0@boostconsulting.com> References: <200204031856.g33Iuqr22751@odiug.zope.com> <024301c1db59$99b801f0$1101a8c0@boostconsulting.com> Message-ID: David> FWIW, that's the way it is in C++, and almost all of the same David> conclusions fall out of it. Also, as distasteful as it may seem David> intellectually, I have yet to hear any complaints about this David> situation in practice... so it seems to work out OK. It seems only fair that I should mention a complaint: The fact that bool is an integral type makes it necessary to have operator const void* rather than operator bool as members of istream and ostream for end-of-file testing. Otherwise, writing ``cin << x'' instead of ``cin >> x'' by mistake would result in converting cin to bool and thence to int, shifting the result right by x bits, and throwing it away. -- Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark From ark@research.att.com Thu Apr 4 04:09:59 2002 From: ark@research.att.com (Andrew Koenig) Date: 03 Apr 2002 23:09:59 -0500 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: References: Message-ID: >> No. That would break backwards compatibility. False==0, and True==1; >> everything else follows from that. (But False is not 0, and True is >> not 1!) Ping> This is a strange distinction to make -- one that is not made in Ping> any other programming language i have ever encountered. I would be very surprised if none of the Lisp family of languages has similar anomalies in the relationship between eq and equal. -- Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark From tim.one@comcast.net Thu Apr 4 04:11:11 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 03 Apr 2002 23:11:11 -0500 Subject: [Python-Dev] Problem with the memory docs Message-ID: The docs for PyMem_Malloc say: Requesting zero bytes returns a non-NULL pointer. I can't guarantee this without insane effort, and our code has never guaranteed this. If pymalloc or the platform allocator can't even allocate one byte in this case, nothing can be done to guarantee it except to return "an address" fabricated out of thin air, and guaranteed not to match any legitimate address. Then realloc and free would have to special-case the snot out of those fabricated addresses to avoid going insane. Note that C does not guarantee malloc(0) will return a non-NULL pointer, even on platforms where malloc(0) *tries* to return a non-NULL pointer. It doesn't have a choice about this: since all non-NULL addresses returned by a conforming C malloc() have to be pairwise distinct so long as none are free()'ed, you can provoke any C implementation into returning NULL for malloc(0) just by calling malloc(0) often enough without doing any free()s. I vote to change the docs. If you vote for something else, you implement it . From zen@shangri-la.dropbear.id.au Thu Apr 4 04:22:52 2002 From: zen@shangri-la.dropbear.id.au (Stuart Bishop) Date: Thu, 4 Apr 2002 14:22:52 +1000 Subject: [Python-Dev] PEP 285: Adding a bool type In-Reply-To: <200203300539.g2U5dAR17743@pcp742651pcs.reston01.va.comcast.net> Message-ID: On Saturday, March 30, 2002, at 04:39 PM, Guido van Rossum wrote: > > Abstract > > This PEP proposes the introduction of a new built-in type, bool, > with two constants, False and True. The bool type would be a > straightforward subtype (in C) of the int type, and the values > False and True would behave like 0 and 1 in most respects (for > example, False==0 and True==1 would be true) except repr() and > str(). All built-in operations that conceptually return a Boolean > result will be changed to return False or True instead of 0 or 1; > for example, comparisons, the "not" operator, and predicates like > isinstance(). On a side note, writing 'i or False and True' performs much better than bool(i) in the test implementation, and flow control statements involving ints are perform much better than flow control using bool. Will there be a performance degredation in the final version, or can this be avoided? > 1) Should this PEP be accepted at all. -0 The only advantage I see is that there is a standard type for jython and other tools that need a boolean type can use. This is also a disadvantage, as all we really have is an object that may magically turn into an int if you are not careful. If you actually need a bool, you will always need to cast it 'just in case'. The only reason there is to modify existing code to return a bool rather than an int is so that repr(1 > 0) == 'True'. Consider: a = getflag() if a is None: print 'Unset' elif a is True: print 'True' elif a is False: print 'False' The above code reads well, but is incorrect unless we are sure what we are going to find in 'a'. The correct way to write the code is exactly like we do currently - adding a bool type adds nothing to Python code. I think of truth in Python is a state. > Most other details of the proposal are pretty much forced by the > backwards compatibility requirement; e.g. True == 1 and > True+1 == 2 must hold, else reams of existing code would break. > > Minor additional issues: > > 4) Should we strive to eliminate non-Boolean operations on bools > in the future, through suitable warnings, so that e.g. True+1 > would eventually (e.g. in Python 3000 be illegal). Personally, > I think we shouldn't; 28+isleap(y) seems totally reasonable to > me. non-Boolean operations should be eliminated eventually, so True + 1 raises a value error and True + True == True. 28+isleap(y) is an abuse of a side effect. If you want to add the return value to an integer, then you should return an integer and name your function correctly. 28+leapdays(y) is correct in my book, or 28+int(isleap(y)) if you don't live in Sweden. > Most languages eventually grow a Boolean type; even C99 (the new > and improved C standard, not yet widely adopted) has one. > > Many programmers apparently feel the need for a Boolean type; most > Python documentation contains a bit of an apology for the absence > of a Boolean type. I've seen lots of modules that defined > constants "False=0" and "True=1" (or similar) at the top and used Many programmers also feel the need for strong static typing in the language too - and I feel that this is intimately related to this PEP. The advantage of a bool type in a language like Java is that if I try to add an integer to a bool, I get a compile time error. Also note that, at least in C and Java, 'and' and 'or' return a boolean value - we can perform a series of boolean operations and know we will end up with 0 or 1 in C, and a boolean in Java. In Python we could end up with anything, and this behavior is relied on as one of the strengths of the language. There is current a concept of true and false as a state, not a value (or as another said, Something or Nothing). > It has been suggested that, in order to satisfy user expectations, > for every x that is considered true in a Boolean context, the > expression x == True should be true, and likewise if x is > considered false, x == False should be true. This is of course > impossible; it would mean that e.g. 6 == True and 7 == True, from > which one could infer 6 == 7. Similarly, [] == False == None > would be true, and one could infer [] == None, which is not the > case. I'm not sure where this suggestion came from; it was made > several times during the first review period. For truth testing > of a value, one should use "if", e.g. "if x: print 'Yes'", not > comparison to a truth value; "if x == True: print 'Yes'" is not > only wrong, it is also strangely redundant. This reads like an illustration of false logic. 'My dog is brown. My cat is brown. Therefore by dog is my cat'. -- Stuart Bishop http://shangri-la.dropbear.id.au/ From fdrake@acm.org Thu Apr 4 04:32:19 2002 From: fdrake@acm.org (Fred L. Drake) Date: Wed, 3 Apr 2002 23:32:19 -0500 (EST) Subject: [Python-Dev] [maintenance doc updates] Message-ID: <20020404043219.912DC28696@beowolf.fdrake.net> The maintenance version of the documentation has been updated: http://python.sourceforge.net/maint21-docs/ Preliminary documentation for Python 2.1.3. From niemeyer@conectiva.com Thu Apr 4 04:42:55 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Thu, 4 Apr 2002 01:42:55 -0300 Subject: [Python-Dev] ACCEPTED: PEP 285 In-Reply-To: <3CAB8C99.232A8571@metaslash.com> References: <200204032232.g33MWqJ28933@odiug.zope.com> <3CAB8C99.232A8571@metaslash.com> Message-ID: <20020404014255.A1388@ibook.distro.conectiva> Hi Neal! > if a is True: print 'True' [...] > test.py:6: Comparisions with True are not necessary Testing if "a is True" is ok. True and False are singletons this is the best way to know if 'a' is really 'True' (and not 1, [0], "foo", etc). This test will be necessary sometimes. > if a == False: print 'False' [...] > test.py:7: Comparisions with False are not necessary This should probably say something more agressive, like "Comparisions with boolean values may be dangerous", since this may give undesirable results (if the user was trying to check for False, and not for 0, [], "", etc). -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From greg@cosc.canterbury.ac.nz Thu Apr 4 05:05:14 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 04 Apr 2002 17:05:14 +1200 (NZST) Subject: [Python-Dev] Re: PEP 279 In-Reply-To: <004101c1db25$197119c0$4066accf@othello> Message-ID: <200204040505.RAA00497@s454.cosc.canterbury.ac.nz> Raymond Hettinger : > enumerate -- I like this one best because it does what it says. But it doesn't. According to the Merriam-Webster dictionary, enumerate means either "to ascertain the number of : COUNT" or "to specify one after another : LIST". The first is what len() does, and the second doesn't require doing anything. Neither of these is what we want. I notice that "numerate" is listed in the thesaurus as a synonym for "enumerate". Maybe it could serve as a compromise between "enumerate" and "number"? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From skip@pobox.com Thu Apr 4 05:26:01 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 3 Apr 2002 23:26:01 -0600 Subject: [Python-Dev] False alarm: big performance hit in the past few days Message-ID: <15531.58345.980563.848951@12-248-41-177.client.attbi.com> I found the problem. You'll be glad to know it wasn't with the Python code. It turned out to be compiler flags. When I was benchmarking on March 31, I was specifically setting OPT to something like -O3 -DSMALL_REQUEST_THRESHOLD=256 Today, I left SMALL_REQUEST_THRESHOLD alone. Since the default optimization level is -O3, I wasn't specifying OPT at all. The default is "-g -O3". With specific setting of OPT to just -O3, I get these performance numbers: cvs up -D 2002-03-31 12:00 This machine benchmarks at 5652.91 pystones/second Pystone(1.1) time for 100000 passes = 17.81 This machine benchmarks at 5614.82 pystones/second Pystone(1.1) time for 100000 passes = 17.88 This machine benchmarks at 5592.84 pystones/second 278.18user 0.39system 5:11.91elapsed 89%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (400major+3942minor)pagefaults 0swaps cvs up -A Pystone(1.1) time for 100000 passes = 17.8 This machine benchmarks at 5617.98 pystones/second Pystone(1.1) time for 100000 passes = 18.04 This machine benchmarks at 5543.24 pystones/second Pystone(1.1) time for 100000 passes = 17.85 This machine benchmarks at 5602.24 pystones/second 278.95user 0.26system 4:51.27elapsed 95%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (404major+3941minor)pagefaults 0swaps so it appears nothing has changed performance-wise since the 31st. Apparently, addition of debugging symbols exacts a pretty significant toll on performance now. I don't recall that being the case in the past. Perhaps this feature is something new with gcc3. Makes me think perhaps -g should be deleted from the default OPT setting. I'm pretty sure one of the reasons Stallman used to give for always including -g in compiles in the old days (when the only GNU programs were, practically speaking, gcc, gdb and emacs) was that you got debugging capability without taking a performance hit. Skip From martin@v.loewis.de Thu Apr 4 05:31:10 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 04 Apr 2002 07:31:10 +0200 Subject: [Python-Dev] New canonical location for PEPs In-Reply-To: <87ofh0xo93.fsf@tleepslib.sk.tsukuba.ac.jp> References: <15530.31727.557290.92540@grendel.zope.com> <20020402233722.H16962@tummy.com> <20020403004949.I16962@tummy.com> <87ofh0xo93.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: "Stephen J. Turnbull" writes: > I know XEmacs's is not perfect, but I'm fairly sure host config > changes at SourceForge and SunSITE.de have been responsible for most > of the attention the webmaster has paid to the auto-update mechanism > over the last two years. Attention has been paid to the lock issue, > but I don't recall the resolution. It appears that all the magic is in /pack/viewcvs/current/loginfo-handler. Adrian, assuming this is a modified version of loginfo-handler: is this available online somewhere? Regards, Martin From jafo@tummy.com Thu Apr 4 05:34:34 2002 From: jafo@tummy.com (Sean Reifschneider) Date: Wed, 3 Apr 2002 22:34:34 -0700 Subject: CVS update of the web-site (was: Re: [Python-Dev] New canonical location for PEPs) In-Reply-To: <87ofh0xo93.fsf@tleepslib.sk.tsukuba.ac.jp>; from stephen@xemacs.org on Thu, Apr 04, 2002 at 10:10:16AM +0900 References: <15530.31727.557290.92540@grendel.zope.com> <20020402233722.H16962@tummy.com> <20020403004949.I16962@tummy.com> <87ofh0xo93.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <20020403223434.K16962@tummy.com> On Thu, Apr 04, 2002 at 10:10:16AM +0900, Stephen J. Turnbull wrote: >one who brought up the lock issue most recently. You may be able to >browse the implementation in CVSROOT at cvs.xemacs.org. If not, I'm >sure Adrian would be willing to show it to you. Thanks for the pointers... I've spoken with the guy who had set this up before, and apparently they used the checkin e-mail to trigger the process (with appropriate lock-files and a rate limit. Considering that the web-site is on a different machine from the CVS repository. Seems like that would work well. Something like: lockfile -1 -r 10 "$LOCKFILE" 2>/dev/null [ "$?" -ne 0 ] && exit 0 sleep 20 # do update here rm -f "$LOCKFILE" Sean -- Read the directions, even if you don't follow them. -- Mary Schmich Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From goodger@users.sourceforge.net Thu Apr 4 05:50:33 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Thu, 04 Apr 2002 00:50:33 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard DocstringFormat In-Reply-To: <013901c1daf0$e04e2b30$ced241d5@hagrid> Message-ID: Fredrik Lundh wrote: > the first time, I got to the following complete nonsense: > > "XML and SGML /.../ are verbose, difficult to type, and > too cluttered to read comfortably as source." > > and having written several books in SGML and XML without > noticing any of those "widely known" problems, I decided that > it wasn't meaningful to continue. Your ellipsis omits all the good things I said about XML/SGML. The rest of the quote is supposed to be read in the context of: 3. Explicit markup (like XML or TeX) is widely considered unreadable by the uninitiated. Where "by the uninitiated" is key. Having written books in SGML/XML, written the "sre" engine, and made many other wonderful contributions, you are obviously not among the uninitiated. I'll make that more explicit. > I've never seen such an arrogant PEP before Is this intentional flamebait? Pot calling the kettle black? I will try to restrain myself. ;-) > the authors clearly have very little experience from the problem > domain (not only writing and maintaining documentation with markup, > but also what makes Python so incredibly usable), You are mistaken. Personally, I have over six years of experience with SGML & XML, doing document analysis, writing DTDs, and writing document processing systems, for English, Japanese, Chinese, and Korean data. Four years of experience with Python. Member of the Doc-SIG for two years, but I've read over much of the six-year archive. I still have much to learn, no doubt. If you can point out any specific areas where the PEP is lacking with regards to the above, I'd be happy to do the research. > yet they want to want to force their new invention down everyone's > throat (hey, we spent a lot of time desiging this, so of course you > shall have to use it) Perhaps you missed the second paragraph of the Abstract? [This PEP is not] an attempt to deprecate pure plaintext docstrings, which are always going to be legitimate. The reStructuredText markup is an alternative for those who want more expressive docstrings. > want to contribute a PEP? sorry, you have to learn a new markup > language. The current standard could easily coexist with the proposed. Whether or not to use the new markup could be left to PEP authors, or decreed. That's a policy decision. > want to fix something in the README? sorry, you have to learn a new > markup language. want to fix a module in the standard library? > sorry, you have to learn a new markup language. it's easy. How about: "Want to add to Python's standard documentation? Sorry, you have to learn a new markup language: TeX." Same thing. But hacking on reStructuredText is much easier than hacking on TeX. In any case, there is no *requirement* to use reStructuredText. If the PEP is accepted, it becomes *a* standard for docstrings, but not *the only* standard. At minimum, plaintext docstrings will remain. > there are only a couple of 100ks of specifications to read and > understand. There's need for a short user introduction, true. I apologize that it wasn't already in place. The beginnings of such an introduction is at http://structuredtext.sourceforge.net/docs/quickstart.txt. This is the first time I've seen a proposal blasted for having *too much* documentation! ;-) How many Python programmers read the entire Language Reference, let alone the Library Reference? Few, I imagine. So what? Heck, had I been told I had to read O'Reilly's "Python Standard Library" cover to cover before I could write my first line of code, I probably never would have begun. > (and while you're at it, get a new keyboard; we don't care much > about people using non-US keyboards...) Can you be specific? > -1. the world doesn't need another markup language. there is > only one markup language that everyone knows, and it's called > HTML. the javadoc folks got it right. this one's all wrong. You don't like it, fine. I don't see much substantive in your posts here or on comp.lang.python; paraphrased, you're saying "this sucks". Care to give any reasoning behind your conclusion? I must have stepped on some sensitive toes to warrant such a reaction! -- David Goodger goodger@users.sourceforge.net Open-source projects: - Python Docstring Processing System: http://docstring.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net - The Go Tools Project: http://gotools.sourceforge.net From goodger@users.sourceforge.net Thu Apr 4 05:52:14 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Thu, 04 Apr 2002 00:52:14 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard DocstringFormat In-Reply-To: Message-ID: Ka-Ping Yee wrote: [Ping] > I'm not against a new markup language, but i do feel that the > specification of the language is just too big. ... > The current specification is about 10000 words; get it down to about > 1000 and i might go for it. I think this is the crux of the issue. Please realize that the project isn't finished yet. The verbose, comprehensive specification docs are there, because that's what was coded against, that's what was used as material for debate on the Doc-SIG and project mailing lists. Work has already begun on an introductory user document (thanks to Richard Jones; see http://structuredtext.sourceforge.net/docs/quickstart.txt). I apologize that this wasn't available before (and I didn't expect the Spanish Inquisition!). > What's with the 32 different section title adornment characters, the > optional overline, and unspecified title styles (order "as > encountered")? And that's just section titles. Do we really need > five kinds of lists? How about the 15 different ways to number > lists? The five ways to do hyperlink targets? And that's not > all... Different strokes for different folks. Based on observation of actual usage, there's a great variety of implicit markup out there. As long as supporting the variations does not introduce ambiguity, I don't see the problem. With any markup, few users make use of all the features. > I can appreciate the desire to come up with something flexible, but > this goes too far for my taste. The markup was developed with the philosophy of "better a rich set of choices than artificial limits". Which features should be omitted? If deemed too rich for Python docstrings (& PEPs etc.), it can be pared back. But since the full spec is out there, I doubt if anyone could come up with a generally acceptable subset. -- David Goodger goodger@users.sourceforge.net Open-source projects: - Python Docstring Processing System: http://docstring.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net - The Go Tools Project: http://gotools.sourceforge.net From goodger@users.sourceforge.net Thu Apr 4 05:59:28 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Thu, 04 Apr 2002 00:59:28 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format In-Reply-To: <15531.10824.991521.687697@magrathea.basistech.com> Message-ID: Tom Emerson wrote: >> - JavaDoc [10]_ >> >> ... However, the only output format that is supported is HTML > > This is patently false: there are Doclets available that convert to a > wide variety of formats. I was unaware. I've modified the text to remove the false statement. Thanks for the heads-up. -- David Goodger goodger@users.sourceforge.net Open-source projects: - Python Docstring Processing System: http://docstring.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net - The Go Tools Project: http://gotools.sourceforge.net From zen@shangri-la.dropbear.id.au Thu Apr 4 06:22:40 2002 From: zen@shangri-la.dropbear.id.au (Stuart Bishop) Date: Thu, 4 Apr 2002 16:22:40 +1000 Subject: [Python-Dev] PEP 285: Adding a bool type In-Reply-To: Message-ID: <5DB5EE2C-4794-11D6-8924-000393031882@shangri-la.dropbear.id.au> On Thursday, April 4, 2002, at 02:22 PM, Stuart Bishop wrote: > On a side note, writing 'i or False and True' performs much better > than bool(i) in the test implementation, and flow control statements > involving ints are perform much better than flow control using bool. > Will there be a performance degredation in the final version, or > can this be avoided? I fixed my test to not do a global lookup on True and flow control is no longer slower. now-+1-if-anyone-cares-but-lets-see-what-another-100-posts-do-ly y'rs -- Stuart Bishop http://shangri-la.dropbear.id.au/ From mhammond@skippinet.com.au Thu Apr 4 06:36:08 2002 From: mhammond@skippinet.com.au (Mark Hammond) Date: Thu, 4 Apr 2002 16:36:08 +1000 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape Message-ID: Hi all, I wrote a cute little Python sidebar for Mozilla/NS6. It allows quick access to bugs/patches and PEPs, and has a search facility using the ActiveState mailinglist archive. It is pretty cute, but suggestions for new features welcome. As it has the PEP and Bugs feature, it is really suited more to python-dev than python-list - hence my mail here :) Check it out at http://starship.python.net/crew/mhammond/mozilla Mark. From adrian@xemacs.org Thu Apr 4 06:44:58 2002 From: adrian@xemacs.org (Adrian Aichner) Date: 04 Apr 2002 08:44:58 +0200 Subject: [Python-Dev] New canonical location for PEPs In-Reply-To: <15531.47142.871230.351524@grendel.zope.com> References: <15530.31727.557290.92540@grendel.zope.com> <20020402233722.H16962@tummy.com> <20020403004949.I16962@tummy.com> <87ofh0xo93.fsf@tleepslib.sk.tsukuba.ac.jp> <15531.47142.871230.351524@grendel.zope.com> Message-ID: <8z84hsid.fsf@smtprelay.t-online.de> >>>>> "Fred" == Fred L Drake, writes: Fred> Stephen J. Turnbull writes: >> I know XEmacs's is not perfect, but I'm fairly sure host config Fred> Close enough for me! ;-) >> Adrian Aichner is the guy to talk to about the >> website experience (eg, admin resources consumed since original >> setup). I believe Martin Buchholz designed the >> mechanism and did the original implementation. Karl Hegbloom >> adapted it for a project he works on, and was the >> one who brought up the lock issue most recently. You may be able to >> browse the implementation in CVSROOT at cvs.xemacs.org. If not, I'm >> sure Adrian would be willing to show it to you. Fred> It doesn't look like the CVSROOT directory is visible in Fred> your ViewCVS installation. I'd be interested in seeing how Hi Fred! It's just not advertised. See http://cvs.xemacs.org/viewcvs.cgi/CVSROOT/ You can check it out anonymously. Fred> you weave the pieces together. The material on python.org See http://www.dk.xemacs.org/About/Website.html Fred> is currently woven together from multiple CVS repositories Fred> (I can think of three, at least!). We have three modules contribute to our website as well: xemacsweb packages xemacs-21.5 (currently) Integration is still under development but our initial implementation works already. I'll document it a bit more when I get a chance. Hope this helps, Adrian Fred> -Fred Fred> -- Fred> Fred L. Drake, Jr. Fred> PythonLabs at Zope Corporation -- Adrian Aichner mailto:adrian@xemacs.org http://www.xemacs.org/ From adrian@xemacs.org Thu Apr 4 06:49:28 2002 From: adrian@xemacs.org (Adrian Aichner) Date: 04 Apr 2002 08:49:28 +0200 Subject: [Python-Dev] New canonical location for PEPs In-Reply-To: References: <15530.31727.557290.92540@grendel.zope.com> <20020402233722.H16962@tummy.com> <20020403004949.I16962@tummy.com> <87ofh0xo93.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <4rishsav.fsf@smtprelay.t-online.de> >>>>> "Martin" == Martin v Loewis writes: Martin> "Stephen J. Turnbull" writes: >> I know XEmacs's is not perfect, but I'm fairly sure host config >> changes at SourceForge and SunSITE.de have been responsible for most >> of the attention the webmaster has paid to the auto-update mechanism >> over the last two years. Attention has been paid to the lock issue, >> but I don't recall the resolution. Hi Martin! Martin> It appears that all the magic is in Martin> /pack/viewcvs/current/loginfo-handler. Adrian, assuming this is a No, /pack/viewcvs/current/loginfo-handler is only used to update a commit database for ViewCVS. We had this working but it's not working since an upgrade to ViewCVS 0.9.2. Martin> modified version of loginfo-handler: is this available online somewhere? See the XEmacs/xemacsweb entry in http://cvs.xemacs.org/viewcvs.cgi/CVSROOT/loginfo?rev=HEAD&content-type=text/vnd.viewcvs-markup Martin> Regards, Martin> Martin Best regards, Adrian -- Adrian Aichner mailto:adrian@xemacs.org http://www.xemacs.org/ From greg@cosc.canterbury.ac.nz Thu Apr 4 06:52:10 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 04 Apr 2002 18:52:10 +1200 (NZST) Subject: [Python-Dev] ANN: Pyrex - a language for writing Python extension modules Message-ID: <200204040652.SAA00676@s454.cosc.canterbury.ac.nz> Folks here may be interested in my latest project: A new language specially designed for writing Python extension modules. Pyrex is more or less Python with C data types added. You can write functions which freely intermix operations on Python and C data, with all Python reference counting and error checking completely automated. An example Pyrex module is shown below. For more information, see: http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/ where you can also download version 0.1 and try it out. I'm very keen to get feedback on this, both to find out whether people think it's useful, and to help me debug it! Oh, yeah, here's the example. It finds prime numbers at C speed. def primes(int kmax): cdef int n, k, i cdef int p[1000] result = [] if kmax > 1000: kmax = 1000 k = 0 n = 2 while k < kmax: i = 0 while i < k and n % p[i] <> 0: i = i + 1 if i == k: p[k] = n k = k + 1 result.append(n) n = n + 1 return result Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From jafo@tummy.com Thu Apr 4 07:09:00 2002 From: jafo@tummy.com (Sean Reifschneider) Date: Thu, 4 Apr 2002 00:09:00 -0700 Subject: [Python-Dev] ANN: Pyrex - a language for writing Python extension modules In-Reply-To: <200204040652.SAA00676@s454.cosc.canterbury.ac.nz>; from greg@cosc.canterbury.ac.nz on Thu, Apr 04, 2002 at 06:52:10PM +1200 References: <200204040652.SAA00676@s454.cosc.canterbury.ac.nz> Message-ID: <20020404000900.L16962@tummy.com> On Thu, Apr 04, 2002 at 06:52:10PM +1200, Greg Ewing wrote: >Folks here may be interested in my latest >project: A new language specially designed for >writing Python extension modules. Neat! Looks like a pretty slick way of making C modules... A couple of things: My system preferred "#!/usr/bin/env python" instead of "#!/bin/env python" in pyrexc. A link to where you can find Plex would be nice on the main Pyrex page. Sean -- "8 out of 10 Starships that took the Folger's challenge didn't know they weren't using Dilithium Crystals." -- Sean and Anne being silly one night. Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From adrian@xemacs.org Thu Apr 4 06:39:15 2002 From: adrian@xemacs.org (Adrian Aichner) Date: 04 Apr 2002 08:39:15 +0200 Subject: [Python-Dev] New canonical location for PEPs In-Reply-To: <87ofh0xo93.fsf@tleepslib.sk.tsukuba.ac.jp> References: <15530.31727.557290.92540@grendel.zope.com> <20020402233722.H16962@tummy.com> <20020403004949.I16962@tummy.com> <87ofh0xo93.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: >>>>> "Stephen" == Stephen J Turnbull writes: >>>>> "Sean" == Sean Reifschneider writes: Sean> There is some reasonable mechanism for [automatically Sean> pushing content from CVS to the Web], as it *IS* done in Sean> some places. Stephen> I know XEmacs's is not perfect, but I'm fairly sure host Stephen> config changes at SourceForge and SunSITE.de have been Make that SunSITE.dk. Stephen> responsible for most of the attention the webmaster has Stephen> paid to the auto-update mechanism over the last two Stephen> years. Attention has been paid to the lock issue, but I I agree. Stephen> don't recall the resolution. I'll have to dig this up. Stephen> We currently keep the website content in CVS at Stephen> SunSITE.de. A CVS commit to the website module triggers SunSITE.dk, not SunSITE.de. Stephen> updates of the SourceForge, Tux, and SunSITE "primary" Stephen> homepages, typically taking 10-15 seconds to propagate to Stephen> all three sites, including a "genpage" run. Take a look at http://www.dk.xemacs.org/logs/?M=D http://www.us.xemacs.org/logs/?M=D http://www.xemacs.org/logs/?M=D for the actual data. Stephen> Adrian Aichner is the guy to talk to Stephen> about the website experience (eg, admin resources Stephen> consumed since original setup). I believe Martin Stephen> Buchholz designed the mechanism and Stephen> did the original implementation. Karl Hegbloom Martin was instrumental to get the ssh magic in the CVSROOT/commit-trigger to work. See http://www.dk.xemacs.org/About/Website.html for an intro. Stephen> adapted it for a project he works Stephen> on, and was the one who brought up the lock issue most Stephen> recently. You may be able to browse the implementation Yes. Stephen> in CVSROOT at cvs.xemacs.org. If not, I'm sure Adrian Stephen> would be willing to show it to you. CVSROOT is not browsable, but you can check it out anonymously: cvs -f -z3 -d :pserver:cvs@cvs.xemacs.org:/pack/xemacscvs checkout CVSROOT See http://www.dk.xemacs.org/Develop/cvsaccess.html Stephen> -- Stephen> Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp Stephen> University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Stephen> Don't ask how you can "do" free software business; Stephen> ask what your business can "do for" free software. -- Adrian Aichner mailto:adrian@xemacs.org http://www.xemacs.org/ From tanzer@swing.co.at Thu Apr 4 07:04:09 2002 From: tanzer@swing.co.at (Christian Tanzer) Date: Thu, 04 Apr 2002 09:04:09 +0200 Subject: [Python-Dev] ACCEPTED: PEP 285 In-Reply-To: Your message of "Wed, 03 Apr 2002 14:47:08 -0800." <3CAB866C.4DF51F72@activestate.com> Message-ID: David Ascher wrote: > > Despite the negative feedback, I've decided to accept the PEP. The > > most important perceived problem is that newbies tend to write > > > > if x =3D=3D True: ... > > > > where they should write > > > > if x: ... > > > > I believe this problem should be solved by education (the Zen master > > hits the student on the head with a stick, and the student gets > > enlightenment) rather than by holding back what I believe will be a > > useful feature. > > Would it not be "relatively" easy to add a compiler-time warning for > most uses of this dangerous idiom? > > At the very least, the boolean type could do the equivalent of: > > def __eq__(self, other): > if __debug__ and other not in (0,1,True,False): > warnings.warn("Testing of equality is best done by asking > objects whether they're true, not comparing with a boolean", > RunTimeWarning) > return int.__eq__(self, other) > > or something similar? I warning about `x =3D=3D True` wouldn't hurt. But your proposed `__eq__`= certainly would -- don't do that please. Equality testing is useful if one wants to check whether two predicates have the same boolean value. -- = Christian Tanzer tanzer@swing.co.= at Glasauergasse 32 Tel: +43 1 876 62 = 36 A-1130 Vienna, Austria Fax: +43 1 877 66 = 92 From tanzer@swing.co.at Thu Apr 4 06:12:54 2002 From: tanzer@swing.co.at (Christian Tanzer) Date: Thu, 04 Apr 2002 08:12:54 +0200 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: Your message of "Wed, 03 Apr 2002 09:37:02 CDT." <200204031437.g33Eb2506920@pcp742651pcs.reston01.va.comcast.net> Message-ID: > Python is not trying to follow the text books. (In fact, maybe I > should say "Python is trying not to follow the text books.") Maybe you could add that gem to the PEP? -- = Christian Tanzer tanzer@swing.co.= at Glasauergasse 32 Tel: +43 1 876 62 = 36 A-1130 Vienna, Austria Fax: +43 1 877 66 = 92 From mal@lemburg.com Thu Apr 4 08:43:07 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 04 Apr 2002 10:43:07 +0200 Subject: [Python-Dev] Re: PEP 285: Adding a bool type References: <2zRcLUAVjXp8EwVv@jessikat.fsnet.co.uk> <200204021340.g32Dekrq025410@ratthing-b246.strakt.com> <3CA9FD45.E6E33129@lemburg.com> <200204022145.g32LjAK20525@odiug.zope.com> <3CAB0BC1.20B90FED@lemburg.com> <016d01c1db1a$fc335620$0900a8c0@spiff> Message-ID: <3CAC121B.AD5192DE@lemburg.com> Fredrik Lundh wrote: > > mal wrote: > > > > How about adding a new pure bool type to the language > > *without* changing any of the old Py_False/Py_True > > code (at first) ?! > > > > Then leave the code in for one release to see how > > people feel about it in real life and enable a warning > > for incorrect usage of comparison arithmetic in a second > > release. The third release would then enable the new > > logic per default. > > from __future__ import boolean ? Hmm, would be nice, but has a flaw: the change wouldn't be limited to one module: it would affect the current instance of the interpreter. Still, a switch like this would be nice to have... probably too late though: Guido has already decided that its better to rush this than come up with a less controversial design. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From mwh@python.net Thu Apr 4 08:51:04 2002 From: mwh@python.net (Michael Hudson) Date: 04 Apr 2002 09:51:04 +0100 Subject: [Python-Dev] Re: Suggested memory API rules for 2.3 In-Reply-To: Tim Peters's message of "Wed, 03 Apr 2002 17:24:02 -0500" References: Message-ID: <2m8z83kft3.fsf@starship.python.net> Tim Peters writes: > [the recommended 2.3+ API] > > PyMem_{Malloc, Realloc, Free} > > PyObject_{Malloc, Realloc, Free} > > PyObject_{New, NewVar} > > [Michael Hudson] > > We should probably provide an example of preprocessor hackery that > > lets you use these interfaces and remain backward compatible as far as > > possible. Maybe even a little .h file which we can suggest extension > > authors bundle with their extensions. > > I would be delighted if somebody else did that. As I age, I find it > increasingly difficult to do more than 103 things at once . I hope you don't mind if I wait a few weeks. Though I guess I wouldn't be aiming for a moving target now, I'll feel safer in a month or so... Cheers, M. -- First time I've gotten a programming job that required a drug test. I was worried they were going to say "you don't have enough LSD in your system to do Unix programming". -- Paul Tomblin -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html From mal@lemburg.com Thu Apr 4 08:53:51 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 04 Apr 2002 10:53:51 +0200 Subject: [Python-Dev] Re: PEP 285: Adding a bool type References: <2zRcLUAVjXp8EwVv@jessikat.fsnet.co.uk> <200204021340.g32Dekrq025410@ratthing-b246.strakt.com> <3CA9FD45.E6E33129@lemburg.com> <200204022145.g32LjAK20525@odiug.zope.com> <3CAB0BC1.20B90FED@lemburg.com> <200204031437.g33Eb2506920@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CAC149F.3F1BA77B@lemburg.com> Guido van Rossum wrote: > > > Right, but my point is that if you want to make booleans first class > > objects in Python, you should consider making them behave as defined > > in the text books rather than trying to make them feel like integers > > but look like truth values. > > I reject this reasoning. Python is not trying to follow the text > books. (In fact, maybe I should say "Python is trying not to follow > the text books.") Uhm, I can't follow you here -- which text books you are talking about here (Python has always tried to conform to standards in most areas). FWIW, I was talking about the text books students use to learn boolean algebra. BTW, I wonder why you didn't follow up on my other comments... -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From vmarangozov@optimay.com Thu Apr 4 08:57:30 2002 From: vmarangozov@optimay.com (Marangozov, Vladimir (Vladimir)) Date: Thu, 4 Apr 2002 10:57:30 +0200 Subject: [Python-Dev] pymalloc killer Message-ID: <464156E0633B154091731247D911C9CF0E0CF7@ge0057excuag01.ags.agere.com> [me] > >... > > In other words, the lastoffset acts as an upper bound watermark. [Tim] > I added a bunch of low-level overviews (arean mgmt, pool > mgmt, block mgmt) as comment blocks over the last week, and noted > the possibility for this in one of them. Another possibility is > to get rid of the runtime divisions, and the pool_header capacity > member, by exploiting that > > pool->capacity = (POOL_SIZE - POOL_OVERHEAD) / size; > > is invariant across all pools with the same szidx; i.e., all > references to pool->capacity *could* be replaced by > capacity[pool->szidx], where "capacity" is a new small (at most > 32 uints) file-static const vector computed at compile-time. The comparison in the "try to extend the free list" code is executed very frequently, so probing a static vector this way will slow down the thing noticeably. I've tried that. We are very much running after speed here. [me] > > I didn't want to do that optimization before, because it would have > > resulted in a bigger pool header and waste of space. Now it's ok. [Tim] > So now the tradeoff is a little muddier: If I skipped the > multiplication optimization, but did the division optimization, > each pool would suddenly gain 8 more bytes (on 32-bit boxes) to use > for blocks. If I did both, the pool_header size would remain as it > is now, "halfway between" multiple-of-8-byte sizes. How do you judge > the tradeoffs now (do only one, or do both)? Do both, despite that the multiplication optimization is more important. (for one division we have #blocks multiplications for a full pool) 1) I think that adding back one uint slot (lastoffset) is not a problem (and we end up with the perfect balance of having a pool header layout with 4 pointers and 4 integers ). 2) Now, with the advent of lastoffset, we can easily get rid off the division as well. For this, we switch to offset calculus for figuring out whether the free list can be extended. This means that 'capacity' becomes 'highoffset'. The 'highoffset' is the highwater not to be exceeded by more than 'size' bytes by 'lastoffest'. a) the test becomes then: if (pool->lastoffset <= pool->highoffset) ... pool->lastoffset += size; pool->freeblock = (block *)pool + \ pool->lastoffset; b) in the pool_init code: pool->lastoffset = POOL_OVERHEAD + size; pool->highoffset = POOL_SIZE - size - size; and you'll need to adjust the code accordingly at the places where you're using 'capacity' per se (and adapt the comments, of course). This should pretty much settle these murky optimizations for now. Cheers, Vladimir From tim.one@comcast.net Thu Apr 4 09:04:20 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 04 Apr 2002 04:04:20 -0500 Subject: [Python-Dev] Re: Suggested memory API rules for 2.3 In-Reply-To: <2m8z83kft3.fsf@starship.python.net> Message-ID: [Michael Hudson] >>> We should probably provide an example of preprocessor hackery that >>> lets you use these interfaces and remain backward compatible as far as >>> possible. Maybe even a little .h file which we can suggest extension >>> authors bundle with their extensions. [Tim] >> I would be delighted if somebody else did that. As I age, I find it >> increasingly difficult to do more than 103 things at once . [Michael] > I hope you don't mind if I wait a few weeks. Though I guess I > wouldn't be aiming for a moving target now, I'll feel safer in a month > or so... I'd wait too, if I were you -- the only thing we've actually finished so far are the hard parts . I hope /F and MAL will help too, since they know more about writing Python extensions in real life than anyone except the Queen Mum. and-now-that-she's-dead-she-probably-knows-everything-ly y'rs - tim From mal@lemburg.com Thu Apr 4 10:47:07 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 04 Apr 2002 12:47:07 +0200 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape References: Message-ID: <3CAC2F2B.1950AD8F@lemburg.com> Mark Hammond wrote: > > Hi all, > I wrote a cute little Python sidebar for Mozilla/NS6. It allows quick > access to bugs/patches and PEPs, and has a search facility using the > ActiveState mailinglist archive. It is pretty cute, but suggestions for new > features welcome. As it has the PEP and Bugs feature, it is really suited > more to python-dev than python-list - hence my mail here :) > > Check it out at http://starship.python.net/crew/mhammond/mozilla Way cool ! Would it be hard to add a hook to the Python search facility as well ? -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From thomas.heller@ion-tof.com Thu Apr 4 12:06:34 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 4 Apr 2002 14:06:34 +0200 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape References: Message-ID: <008a01c1dbd1$2ac8e020$e000a8c0@thomasnotebook> > Hi all, > I wrote a cute little Python sidebar for Mozilla/NS6. It allows quick > access to bugs/patches and PEPs, and has a search facility using the > ActiveState mailinglist archive. It is pretty cute, but suggestions for new > features welcome. As it has the PEP and Bugs feature, it is really suited > more to python-dev than python-list - hence my mail here :) > > Check it out at http://starship.python.net/crew/mhammond/mozilla Looks great - may help me to finally switch from IE to Mozilla ;-) The killer feature for me would be a search box giving capabilities similar to the Index or Search features in the HTMLHelp version of the Python docs. Would that be possible? Thomas From Paul.Moore@atosorigin.com Thu Apr 4 13:08:16 2002 From: Paul.Moore@atosorigin.com (Moore, Paul) Date: Thu, 4 Apr 2002 14:08:16 +0100 Subject: [Python-Dev] Pymalloc and backward compatibility Message-ID: <714DFA46B9BBD0119CD000805FC1F53B01B5B2A4@UKRUX002.rundc.uk.origin-it.com> I've been watching the pymalloc discussions going on recently, and I'm concerned that there may be some fairly major compatibility changes coming. I'd like to check that I understand the intention correctly. To provide some background - I wrote the Python interface for the Vim editor, a long time ago. At the time, Python 1.4 was the current version. I was *very* new to Python at the time, and this was my first real C extension (and is still my biggest - I haven't had much of a need to write my own C extensions since...) I didn't particularly understand the details of the API, so I basically just copied code from existing samples, and was pleased at how easily it all worked. The code has survived essentially unchanged through to Python 2.2, and still works fine. However, I'd be surprised if it doesn't take some fairly extreme liberties (mixing API "families", relying on all Del/Free calls mapping to the same underlying free(), etc). I can't even risk "fixing" it as I don't have any way of checking which API calls appeared in which version (I recall a message in one of the pymalloc threads to the effect that 1.4-compatible code was bound to include some of these bugs, because certain APIs weren't available in 1.4) OK, so maybe it's time to give up on Python 1.4 compatibility. But ideally, I'd prefer to go for an approach which leaves the code used under Python 2.2 and earlier completely unchanged, and only adds the minimum necessary for newer versions. Longer term, it might be worth considering upgrading the code to use some of the newer features in the Python API, but in all honesty that's not a great priority for me (and unlikely to be even remotely on the cards until I can realistically desupport everything before 2.0...) It seems to me that there are two options open for me: 1. The Python API from 1.4 is still guaranteed to work unchanged (even given the liberties mentioned above). In that case, do nothing. 2. There are changes required. In that case, make them, protected by python API checks (annoying ifdef fun). While (1) is superficially nicer from my point of view, it just defers the problem - at *some* point, the backward compatibility load becomes too much. So the 1.4-compatible code is deprecated, at best. In that case, I'd rather change things now, while the issues are fresh. The problem with (2) is that my immediate reaction is to do something like (pseudocode because I'm too lazy to look up the real API names and calls) #ifdef PYTHON_VERSION >= 2.3 PyObject *myObj = Py_WhizzyNewAllocator(); #else PyObject *myObj = Py_CruftyOldAllocator(); #endif That works fine - and looks OK from a maintenance standpoint, as long as the new API really is new. If the new API merely blesses existing calls as the "official right thing", the above code is going to look odd to a future maintainer (Why is the call to Py_WhizzyNewAllocator only enabled for 2.3 onwards? That API call was available from 1.5.2...) So I think what I'm saying is that my preference is for a completely new set of allocation APIs for Python 2.3, with all old APIs deprecated (and scheduled for removal at some time in the future). This should be clearly documented in the manual, including a list of all deprecated APIs to allow extension writers to grep for code that needs changing. Sample ifdef-protected type code like the above, for people like me who tend to treat all this as black magic anyway, would probably be a nice bonus. If this is what was planned anyway, just ignore me. I'll go back to sleep now....... Paul. From jason@tishler.net Thu Apr 4 13:20:53 2002 From: jason@tishler.net (Jason Tishler) Date: Thu, 04 Apr 2002 08:20:53 -0500 Subject: [Python-Dev] RE: POSIX thread code and Cygwin In-Reply-To: <20020323160823.GB1696@tishler.net> References: <20020323160823.GB1696@tishler.net> Message-ID: <20020404132053.GA932@tishler.net> On Sat, Mar 23, 2002 at 11:08:23AM -0500, Jason Tishler wrote: > I agree with Tim (and Martin) that this is a Cygwin (i.e., newlib) > issue and should not be "fixed" or worked around in Python. Would you > be willing to submit the attached (untested) patch to newlib after giving > it a spin? Just a FYI... This issue has been resolved in Cygwin (i.e., newlib). If interested, see the following for details: http://sources.redhat.com/ml/newlib/2002/msg00122.html I will (finally) release a threaded Cygwin Python as soon as Cygwin 1.3.11 is released. Jason From mal@lemburg.com Thu Apr 4 13:27:14 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 04 Apr 2002 15:27:14 +0200 Subject: [Python-Dev] Freeze and shared modules References: <3CA9E805.AE012428@lemburg.com> Message-ID: <3CAC54B2.C2F5B76E@lemburg.com> "Martin v. Loewis" wrote: > > "M.-A. Lemburg" writes: > > > Does anyone know which compiler/linker options have to be specified > > to make frozen Python interpreters correctly import shared modules ? > > LINKFORSHARED should be the right set of options. Thanks for the hint: freeze.py generates a Makefile which doesn't use this symbol for linking. Adding it by hand fixes this problem. I'll checkin a fix for it. > > Many of the mxCGIPython are running into this problem (the default > > settings in Python or freeze seem to be wrong on this account) and > > it got even worse with Python 2.2 where even the standard modules > > are built shared rather than statically linked. > > Yes, building all extension modules as shared libraries is evil for > freezing applications. I wish Linux distributors would recognize this > and write proper Setup files, instead of relying on the default > mechanism. mxCGIPython comes with its own Modules/Setup file, so the problem is slightly less important. I agree though that at least the standard modules which don't require additional setup.py configuration should be turned on again per default in Modules/Setup. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From neal@metaslash.com Thu Apr 4 13:35:05 2002 From: neal@metaslash.com (Neal Norwitz) Date: Thu, 04 Apr 2002 08:35:05 -0500 Subject: [Python-Dev] ACCEPTED: PEP 285 References: <200204032232.g33MWqJ28933@odiug.zope.com> <3CAB866C.4DF51F72@activestate.com> <200204032305.g33N5nm29961@odiug.zope.com> <3CAB9253.F5FFD2F7@activestate.com> <200204040059.g340x9b08176@pcp742651pcs.reston01.va.comcast.net> <3CABA820.735A9958@metaslash.com> <200204040129.g341TFJ09380@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CAC5689.CAF412C2@metaslash.com> Guido van Rossum wrote: > Hey! Who told you you could have the keys to the time machine! Awwwwwww, Daaadddddd. I just wanted to take a test drive. Who's it gonna hurt? (heard in the van Rossum house around 2017) From neal@metaslash.com Thu Apr 4 13:37:43 2002 From: neal@metaslash.com (Neal Norwitz) Date: Thu, 04 Apr 2002 08:37:43 -0500 Subject: [Python-Dev] ACCEPTED: PEP 285 References: <200204032232.g33MWqJ28933@odiug.zope.com> <3CAB8C99.232A8571@metaslash.com> <200204040052.g340qGL07447@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CAC5727.D4B8A66B@metaslash.com> Guido van Rossum wrote: > > I wonder if you should add warnings for *any* spelling, or at least Done. > But 'is True' can be a useful test. The newbies we're trying to catch > in verboten activities are using ==. Done, 'is' and 'is not' do not generate a warning. Also fixed the spelling mistake and made the warning harsher: Comparisons with True are not necessary and may not work as expected Neal From gsw@agere.com Thu Apr 4 13:40:32 2002 From: gsw@agere.com (Gerald S. Williams) Date: Thu, 4 Apr 2002 08:40:32 -0500 Subject: [Python-Dev] ACCEPTED: PEP 285 In-Reply-To: Message-ID: Interesting how the posts seem to be actually picking up. :-) One concern about testing for "== True" is that it might push newbies into using "is True" instead. Generally, "is True" is even more dangerous than "== True" unless you're testing to see if it's a boolean at all (in which case there should be a test for "is False"), or doing something rather esoteric. -Jerry -O Gerald S. Williams, 55A-134A-E : mailto:gsw@agere.com O- -O AGERE SYSTEMS, 6755 SNOWDRIFT RD : office:610-712-8661 O- -O ALLENTOWN, PA, USA 18106-9353 : mobile:908-672-7592 O- From pobrien@orbtech.com Thu Apr 4 13:45:41 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Thu, 4 Apr 2002 07:45:41 -0600 Subject: [Python-Dev] Re: PEP 279 In-Reply-To: <200204040505.RAA00497@s454.cosc.canterbury.ac.nz> Message-ID: [Greg Ewing] > > Raymond Hettinger : > > > enumerate -- I like this one best because it does what it says. > > But it doesn't. According to the Merriam-Webster dictionary, > enumerate means either "to ascertain the number of : COUNT" > or "to specify one after another : LIST". Maybe a more generous dictionary would help. <0.5 wink> http://www.dictionary.com/search?q=enumerate Included in the Webster's Revised Unabridged Dictionary are phrases such as: "to tell by numbers; to count over, or tell off one after another; to mention one by one; to name over; to make a special and separate account of" Sounds close enough to me. Extremely close, actually. And I don't think we'll find anything better. --- Patrick K. O'Brien Orbtech From David Abrahams" Message-ID: <071201c1dbe1$3dc96d60$1101a8c0@boostconsulting.com> ----- Original Message ----- From: "Patrick K. O'Brien" > Maybe a more generous dictionary would help. <0.5 wink> > > http://www.dictionary.com/search?q=enumerate > > Included in the Webster's Revised Unabridged Dictionary are phrases such as: > > "to tell by numbers; to count over, or tell off one after another; to > mention one by one; to name over; to make a special and separate account of" > > Sounds close enough to me. Extremely close, actually. I agree. > And I don't think > we'll find anything better. Enumerat's OK with me, but I want to put in another plug for itemize(), which is in the thesaurus as a synonym. Think of what an itemized list is, for example: 1. foo 2. bar 3. baz 4. ... -Dave From guido@python.org Thu Apr 4 14:35:15 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 04 Apr 2002 09:35:15 -0500 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: Your message of "Thu, 04 Apr 2002 10:53:51 +0200." <3CAC149F.3F1BA77B@lemburg.com> References: <2zRcLUAVjXp8EwVv@jessikat.fsnet.co.uk> <200204021340.g32Dekrq025410@ratthing-b246.strakt.com> <3CA9FD45.E6E33129@lemburg.com> <200204022145.g32LjAK20525@odiug.zope.com> <3CAB0BC1.20B90FED@lemburg.com> <200204031437.g33Eb2506920@pcp742651pcs.reston01.va.comcast.net> <3CAC149F.3F1BA77B@lemburg.com> Message-ID: <200204041435.g34EZFa10532@pcp742651pcs.reston01.va.comcast.net> [MAL] > > > Right, but my point is that if you want to make booleans first class > > > objects in Python, you should consider making them behave as defined > > > in the text books rather than trying to make them feel like integers > > > but look like truth values. [me] > > I reject this reasoning. Python is not trying to follow the text > > books. (In fact, maybe I should say "Python is trying not to follow > > the text books.") [MAL] > Uhm, I can't follow you here -- which text books you are talking about > here (Python has always tried to conform to standards in most areas). The textbooks that say that a Boolean value does not support arithmetic operations. In tis discussion several people have tried to argue for such bools and claimed that that is the textbook definition. > FWIW, I was talking about the text books students use to learn boolean > algebra. I don't think Boolean algebra is a big subject. I bet that most people only learn about it explicitly when they learn computer programming. > BTW, I wonder why you didn't follow up on my other comments... This thread is way too long to comment on everything. I probably felt that I'd already responded to the same issue in a different message. --Guido van Rossum (home page: http://www.python.org/~guido/) From pedroni@inf.ethz.ch Thu Apr 4 14:33:48 2002 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Thu, 4 Apr 2002 16:33:48 +0200 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format Message-ID: <014301c1dbe5$bbbffdc0$6d94fea9@newmexico> mmh, Q: can we see the html doc produced starting from all the modules of a minimal example package? for the rest: [from PEP] 1. To establish a standard docstring format by attaining "accepted" status (Python community consensus; BDFL pronouncement). Once reStructuredText is a Python standard, all effort can be focused on tools instead of arguing for a standard. Python needs a standard set of documentation tools. Really? establish in what sense? BDFL blessing is not sufficient in this case, or is it? especially since this is gonna be a nop for the std lib, and the std doc (for a long moment). Once a standard exists, people will start to use it, and momentum will inevitably gather. This is rather naive. The only thing I found relevant in the PEP are (does this fact means something?): - Markup that isolates a Python identifier: interpreted text. - Markup that isolates a Python identifier and specifies its type: interpreted text with roles. An approach (maybe just even a presentation approach) that focus on the relevant and minimality, will be easier to push down people throat, from the Q&A and the PEP it seems that the PEP is the result of a motivated self-selected group with difficulty to reach consensus and so neglecting minimality... >From the PEP: [[Are these in fact the goals of the Doc-SIG members? Anything to add?]] Throughout the existence of the Doc-SIG, consensus on a single standard docstring format has never been reached. so is this just your proposal? how should we parse that? To be honest the point is not what the goal of the Doc-SIG are, but what the PEP can do for us. My experience with JavaDoc is that people maybe will do the little effort required, if they know: I do this little markup and I get this set of useful htmls that I can put online, e.g. [I'm not arguing in favor of JavaDoc, because I know that plain-text-resemblance is a goal for python docstrings and not for JavaDoc, on the other hand the PEP make some strectched assumptions about what is readability in source code and the fact that a 'apropos' function doing the right stripping cannot be more than OK for interactive use of the __doc__,] The PEP focus seems out of focus: are you asking whether a brand-new rich formatting markup can be used for rich formatting? ye, then what... it can be easely hated too. IOW I mean that the motivation points 1.-4 in Rationale, do not constitute a crystal-clear rationale, I would argue what kind of structure 2. is talking about and it seems that the PEP goes well beyond the kind of structure that the average programmer need and OTOH it seems it has anything to do with the structure of the doc produced by extract-to-documentation tools: what happens if I put 3 level of titles in the doc of a method? oops, ye the abstract says that the PEP is not concerned with this [after some more tought] Now, I see the issue is reST vs. HTML and not JavaDoc vs. reST. Ahaha. So the PEP can only be judged once we can see the reST-ed module -> tex for the std lib doc trip, or is even not about that? the goal as stated are rich docstrings (for what?) and PEPs... I'm puzzled, the PEP does not even refer or cite PEP 257, which OTOH seem also not to address completly the issue of doc-extraction tools interaction. Honestly, PEP 256,257,258, PEP 287 where are we going? One could even argue that the effort necessary to implement reST, is wasted wrt to the goal of having a packages -> useful doc output chain (?), Generality is good but a design driven but at least by *one* *definite* *kind* *of* *output* would not be a bad thing either... I don't see a clear picture of the input -> output relation in all the PEPs, Maybe goal 1 should be honestly rephrased: 1. To establish a standard docstring format by attaining... => 1. To establish a standard docstring *rich formatting* format *for the interested* by attaining ( Btw, many people will not react simply because they don't care and are not going to use this anyway, but some people already "hate" the PEP, that's not a good sign, and BDFL support is not sufficient here IMHO, you got at least the politics wrong IMHO) regards. From nas@python.ca Thu Apr 4 14:39:11 2002 From: nas@python.ca (Neil Schemenauer) Date: Thu, 4 Apr 2002 06:39:11 -0800 Subject: [Python-Dev] Re: Suggested memory API rules for 2.3 In-Reply-To: ; from tim.one@comcast.net on Wed, Apr 03, 2002 at 06:47:39PM -0500 References: <20020403144853.A1509@glacier.arctrix.com> Message-ID: <20020404063910.A4880@glacier.arctrix.com> Tim Peters wrote: > we promise that PyObject_GC_Del can be used in contexts requiring > function designators. People will have to cast to PyObject* when calling PyObject_GC_Del. I guess that will be consistent with the other _Free and _Del functions. Neil From barry@zope.com Thu Apr 4 14:49:22 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 4 Apr 2002 09:49:22 -0500 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape References: Message-ID: <15532.26610.113900.95161@anthem.wooz.org> >>>>> "MH" == Mark Hammond writes: MH> Hi all, I wrote a cute little Python sidebar for Mozilla/NS6. MH> It allows quick access to bugs/patches and PEPs, and has a MH> search facility using the ActiveState mailinglist archive. It MH> is pretty cute, but suggestions for new features welcome. As MH> it has the PEP and Bugs feature, it is really suited more to MH> python-dev than python-list - hence my mail here :) MH> Check it out at MH> http://starship.python.net/crew/mhammond/mozilla Very cool. I might actually start /using/ the sidebar instead of just hiding it out of the way. -Barry From guido@python.org Thu Apr 4 14:54:08 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 04 Apr 2002 09:54:08 -0500 Subject: [Python-Dev] Problem with the memory docs In-Reply-To: Your message of "Wed, 03 Apr 2002 23:11:11 EST." References: Message-ID: <200204041454.g34Es8210623@pcp742651pcs.reston01.va.comcast.net> > The docs for PyMem_Malloc say: > > Requesting zero bytes returns a non-NULL pointer. > > I can't guarantee this without insane effort, and our code has never > guaranteed this. If pymalloc or the platform allocator can't even allocate > one byte in this case, nothing can be done to guarantee it except to return > "an address" fabricated out of thin air, and guaranteed not to match any > legitimate address. Then realloc and free would have to special-case the > snot out of those fabricated addresses to avoid going insane. Note that C > does not guarantee malloc(0) will return a non-NULL pointer, even on > platforms where malloc(0) *tries* to return a non-NULL pointer. It doesn't > have a choice about this: since all non-NULL addresses returned by a > conforming C malloc() have to be pairwise distinct so long as none are > free()'ed, you can provoke any C implementation into returning NULL for > malloc(0) just by calling malloc(0) often enough without doing any free()s. > > I vote to change the docs. If you vote for something else, you implement it > . The whole post is a joke, right? You just mean that it can still return NULL when it runs out of memory, so the docs should say that? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Apr 4 14:56:30 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 04 Apr 2002 09:56:30 -0500 Subject: [Python-Dev] PEP 285: Adding a bool type In-Reply-To: Your message of "Thu, 04 Apr 2002 14:22:52 +1000." References: Message-ID: <200204041456.g34EuUU10634@pcp742651pcs.reston01.va.comcast.net> > On a side note, writing 'i or False and True' performs much better > than bool(i) in the test implementation, and flow control statements > involving ints are perform much better than flow control using bool. > Will there be a performance degredation in the final version, or can > this be avoided? That surprises me. Can you show your benchmark? (I don't have the time to respond to the rest of your arguments, which have been brought up and responded to enough times in this discussion.) --Guido van Rossum (home page: http://www.python.org/~guido/) From gmcm@hypernet.com Thu Apr 4 15:05:43 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Thu, 4 Apr 2002 10:05:43 -0500 Subject: [Python-Dev] Re: PEP 285: Adding a bool type In-Reply-To: <200204041435.g34EZFa10532@pcp742651pcs.reston01.va.comcast.net> References: Your message of "Thu, 04 Apr 2002 10:53:51 +0200." <3CAC149F.3F1BA77B@lemburg.com> Message-ID: <3CAC2577.18187.47F50BD1@localhost> [Guido] > The textbooks that say that a Boolean value does > not support arithmetic operations. In tis > discussion several people have tried to argue for > such bools and claimed that that is the textbook > definition. Odd. I recall learning that they support addition (identity element False; operation spelled "or") and multiplication (identity element True; operation spelled "and"). So maybe the distinction is the word "arithmetic". Alas, my local library doesn't seem to have a copy of Frege . -- Gordon http://www.mcmillan-inc.com/ From ark@research.att.com Thu Apr 4 15:16:36 2002 From: ark@research.att.com (Andrew Koenig) Date: 04 Apr 2002 10:16:36 -0500 Subject: [Python-Dev] PEP 285: Adding a bool type In-Reply-To: <200204041456.g34EuUU10634@pcp742651pcs.reston01.va.comcast.net> References: <200204041456.g34EuUU10634@pcp742651pcs.reston01.va.comcast.net> Message-ID: >> On a side note, writing 'i or False and True' performs much better >> than bool(i) in the test implementation, and flow control statements >> involving ints are perform much better than flow control using bool. >> Will there be a performance degredation in the final version, or can >> this be avoided? Guido> That surprises me. Can you show your benchmark? Doesn't 'i or False and True' always evaluate to 'i', rather than to 'bool(i)' because 'and' binds more tighly than 'or'? What's the corresponding performance measure for 'i and True or False'? -- Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark From nas@python.ca Thu Apr 4 15:20:50 2002 From: nas@python.ca (Neil Schemenauer) Date: Thu, 4 Apr 2002 07:20:50 -0800 Subject: [Python-Dev] Pymalloc and backward compatibility In-Reply-To: <714DFA46B9BBD0119CD000805FC1F53B01B5B2A4@UKRUX002.rundc.uk.origin-it.com>; from Paul.Moore@atosorigin.com on Thu, Apr 04, 2002 at 02:08:16PM +0100 References: <714DFA46B9BBD0119CD000805FC1F53B01B5B2A4@UKRUX002.rundc.uk.origin-it.com> Message-ID: <20020404072050.B4880@glacier.arctrix.com> Moore, Paul wrote: > I've been watching the pymalloc discussions going on recently, and I'm > concerned that there may be some fairly major compatibility changes > coming. We are trying to avoid that. > The code has survived essentially unchanged through [Python 1.4] to > Python 2.2, and still works fine. This is a great test case. Can you provide a URL to the code? > It seems to me that there are two options open for me: > > 1. The Python API from 1.4 is still guaranteed to work unchanged (even given > the liberties mentioned above). In that case, do nothing. > 2. There are changes required. In that case, make them, protected by python > API checks (annoying ifdef fun). As long as you don't use free() to deallocate objects then you are safe. Python 1.4 has PyMem_DEL. You can use a macro like this: #if PY_VERSION_HEX < 0x01060000 #define PyObject_New PyObject_NEW #define PyObject_Del PyMem_DEL #endif and use PyObject_New and PyObject_Del to manage object memory. Neil From skip@pobox.com Thu Apr 4 15:23:47 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 4 Apr 2002 09:23:47 -0600 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape In-Reply-To: References: Message-ID: <15532.28675.583270.786889@12-248-41-177.client.attbi.com> Mark> I wrote a cute little Python sidebar for Mozilla/NS6. I saw all the other responses so far, and while it looks cute, I can only think of two reasons to use Mozilla/NS6: testing for compatibility and escape from MS. Writing code that is specific to any browser seems counterproductive to me. Skip From nas@python.ca Thu Apr 4 15:25:06 2002 From: nas@python.ca (Neil Schemenauer) Date: Thu, 4 Apr 2002 07:25:06 -0800 Subject: [Python-Dev] Pymalloc and backward compatibility In-Reply-To: <20020404072050.B4880@glacier.arctrix.com>; from nas@python.ca on Thu, Apr 04, 2002 at 07:20:50AM -0800 References: <714DFA46B9BBD0119CD000805FC1F53B01B5B2A4@UKRUX002.rundc.uk.origin-it.com> <20020404072050.B4880@glacier.arctrix.com> Message-ID: <20020404072506.C4880@glacier.arctrix.com> Neil Schemenauer wrote: > You can use a macro like this: > > #if PY_VERSION_HEX < 0x01060000 > #define PyObject_New PyObject_NEW > #define PyObject_Del PyMem_DEL > #endif I just realized that this may be unclear. You don't _need_ to use the above macro to make your code work across versions. As long as you use PyMem_DEL and not free() to deallocate objects the code will work. Neil From thomas.heller@ion-tof.com Thu Apr 4 15:35:15 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 4 Apr 2002 17:35:15 +0200 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) References: <3CAC2F2B.1950AD8F@lemburg.com> Message-ID: <018e01c1dbee$51b1c9f0$e000a8c0@thomasnotebook> This is a multi-part message in MIME format. ------=_NextPart_000_018B_01C1DBFF.14E37940 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit > Would it be hard to add a hook to the Python search facility as > well ? Inspired by Mark's sidebar I spend a little time to implement a small script able to search the Python docs. Since it is very small, I post it here completely. I'm interested in feedback: Is it usefull at all? Should something like this go into the Python distribution? How can it be improved? What should be the user interface - currently it is command line only and usable from the interactive interpreter programmatically. The script downloads the index pages of the Python documentation (api, lib, and ref sections), and parses the HTML code to find all the links. If only one link is found, it is displayed directly with the webbrowser module, if more links are found a temporary page is created showing all these links and displayed. Command line usage: python pyhelp.py [version] version must be 2.0, 2.1, 2.2, or devel topic is the keyword to search for. Examples to try: python pyhelp.py 2.2 Py_Initialize python pyhelp.py Py_INCREF python pyhelp.py __i It is a little bit slow, because it does no caching. Regards, Thomas ------=_NextPart_000_018B_01C1DBFF.14E37940 Content-Type: text/plain; name="pyhelp.py" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="pyhelp.py" import htmllib, formatter, string=0A= import urllib, webbrowser, tempfile, os=0A= import sys=0A= =0A= DOCMAP =3D {=0A= "2.0": "http://www.python.org/doc/2.0/",=0A= "2.1": "http://www.python.org/doc/2.1/",=0A= "2.2": "http://www.python.org/doc/2.2/",=0A= # Can also use local documentation!=0A= ## "2.2": "file:c:/python22/doc/",=0A= "devel": "http://www.python.org/dev/doc/devel/"=0A= }=0A= INDEXPAGE =3D "genindex.html" # Only valid for 2.0 and above=0A= SECTIONS =3D "api/ ref/ lib/".split()=0A= =0A= # modified from an example in the eff-bot guide to the Python Library...=0A= class Parser(htmllib.HTMLParser):=0A= def __init__(self, verbose=3D0):=0A= self.anchors =3D {}=0A= f =3D formatter.NullFormatter()=0A= htmllib.HTMLParser.__init__(self, f, verbose)=0A= self.last_text =3D ""=0A= =0A= def anchor_bgn(self, href, name, type):=0A= self.save_bgn()=0A= self.anchor =3D href=0A= =0A= def anchor_end(self):=0A= text =3D string.strip(self.save_end())=0A= if text =3D=3D "[Link]" and self.last_text:=0A= text =3D self.last_text=0A= if self.anchor and text:=0A= self.anchors[text] =3D self.anchors.get(text, []) + = [self.anchor]=0A= self.last_text =3D text=0A= =0A= =0A= def help(topic, version=3D"2.2"):=0A= baseurl =3D DOCMAP[version]=0A= =0A= v =3D []=0A= =0A= for sec in SECTIONS:=0A= file =3D urllib.urlopen(baseurl + sec + INDEXPAGE)=0A= html =3D file.read()=0A= file.close()=0A= =0A= p =3D Parser()=0A= p.feed(html)=0A= p.close()=0A= =0A= for key, urls in p.anchors.items():=0A= if key.startswith(topic):=0A= for url in urls:=0A= v.append((sec + url, "%s %s" % (sec, key)))=0A= =0A= =0A= if len(v) =3D=3D 0:=0A= print "Not found"=0A= elif len(v) =3D=3D 1: # only one topic found, display directly=0A= webbrowser.open(baseurl + v[0][0])=0A= else:=0A= # create a temporary HTML page displaying links to the=0A= # search results. Unfortunately the file cannot be deleted,=0A= # because it may still be needed by the browser.=0A= # Hm. tempfile.tmpdir is None on windows. Why doesn't it use=0A= # the TMP or TEMP env vars?=0A= path =3D os.path.join(tempfile.tempdir or "", "~results~.html")=0A= file =3D open(path, "w")=0A= file.write("

Search results for '%s':

\n" % topic)=0A= for loc, topic in v:=0A= url =3D baseurl + loc=0A= file.write("

%s: %s

\n" % \=0A= (topic, url, url))=0A= file.close()=0A= webbrowser.open("file:%s" % path)=0A= =0A= if __name__ =3D=3D '__main__':=0A= if len(sys.argv) =3D=3D 3:=0A= version =3D sys.argv[1]=0A= topic =3D sys.argv[2]=0A= elif len(sys.argv) =3D=3D 2:=0A= version =3D "2.2"=0A= topic =3D sys.argv[1]=0A= else:=0A= print "Usage: %s [version] topic" % sys.argv[0]=0A= sys.exit(1)=0A= =0A= help(topic, version)=0A= ------=_NextPart_000_018B_01C1DBFF.14E37940-- From loewis@informatik.hu-berlin.de Thu Apr 4 15:47:41 2002 From: loewis@informatik.hu-berlin.de (Martin v. =?iso-8859-1?q?L=F6wis?=) Date: Thu, 4 Apr 2002 17:47:41 +0200 (CEST) Subject: [Python-Dev] Updating to autoconf 2.5x Message-ID: <200204041547.g34Flflv007093@paros.informatik.hu-berlin.de> Folks, autoconf maintainers have advanced autoconf development over the last few years in slightly incompatible ways; as a result, the have produced autoconf 2.50 and successors. The generated autoconf scripts vary largely in their text, and slightly in their behaviour; autoconf 2.50 offers many additional features, also, not all configure.in scripts work with autoconf 2.50. Access to autoconf 2.13 with Linux distributions is becoming a problem, since some Linux distributors only offer 2.5x packages. So the question arises whether Python's configure should update to 2.50. In rev 1.268, I had ported configure.in to support autoconf 2.50, but have continued to use autoconf 2.13 to regenerate it. If all regular editors of configure.in have autoconf 2.5x available, or can install it, I'd propose to bump the AC_PREREQ to 2.50. For those of you who'd then need both autoconf versions available, I can recommend the Debian wrapper; this is a perl script that invokes either autoconf2.13 or autoconf2.50, depending on several criteria (configure.in or configure.ac? AC_PREREQ?). If there is interest, I can post instructions on how to install both versions side-by-side. So what do you think? Regards, Martin From Paul.Moore@atosorigin.com Thu Apr 4 15:54:59 2002 From: Paul.Moore@atosorigin.com (Moore, Paul) Date: Thu, 4 Apr 2002 16:54:59 +0100 Subject: [Python-Dev] Pymalloc and backward compatibility Message-ID: <714DFA46B9BBD0119CD000805FC1F53B01B5B2A6@UKRUX002.rundc.uk.origin-it.com> From: Neil Schemenauer [mailto:nas@python.ca] >> The code has survived essentially unchanged through >> [Python 1.4] to Python 2.2, and still works fine. > >This is a great test case. Can you provide a URL to the >code? I'm not sure it's downloadable separately, but it's in the sources for Vim (see www.vim.org), file src/if_python.c. If I can, I'll have a go at testing it against 2.3 once the pymalloc situation settles down. I don't normally build Python from source, and my CVS access is flaky at best, so I won't promise anything until after the first alpha, though... :-) > As long as you don't use free() to deallocate objects then > you are safe. > Python 1.4 has PyMem_DEL. You can use a macro like this: > > #if PY_VERSION_HEX < 0x01060000 > #define PyObject_New PyObject_NEW > #define PyObject_Del PyMem_DEL > #endif > > and use PyObject_New and PyObject_Del to manage object memory. Some form of backward compatibility boilerplate which incorporated this would probably be of use in allowing people to migrate - it might save everyone writing their own. One thing that worries me (probably unnecessarily) is that this alters the code on Python 1.6 - 2.2. While there's no reason this should break anything, I don't have the means to test the change. And if I used a version test for 2.3, that would look strange (as I mentioned) because the APIs have been around since before then. But I'm working from the basis that I would support some form of tidying up which *did* result in breakage - whether this happens now or later is up for debate (personally, I'm from the "get it over with" school). From that POV, I don't mind being forced to use the "right" calls, I just want to avoid breaking the code for versions I no longer have any way of testing (but others do still use) while making the backward compatibility related code very clear (so that when I finally do get fed up of believing I support 1.4, I can strip it out cleanly). I think I've talked myself into something like your code, and taking the chance of causing problems with older versions. A table of which API call appeared in which version would be useful to help with this (the current API documentation doesn't include "historical" information like this). Thanks for the comments, I'll go back to lurking now. Paul. From skip@pobox.com Thu Apr 4 16:04:35 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 4 Apr 2002 10:04:35 -0600 Subject: [Python-Dev] Updating to autoconf 2.5x In-Reply-To: <200204041547.g34Flflv007093@paros.informatik.hu-berlin.de> References: <200204041547.g34Flflv007093@paros.informatik.hu-berlin.de> Message-ID: <15532.31123.535297.863287@12-248-41-177.client.attbi.com> Martin> If all regular editors of configure.in have autoconf 2.5x Martin> available, or can install it, I'd propose to bump the AC_PREREQ Martin> to 2.50. I believe 2.50 had serious bugs and was superceded rather quickly by the autoconf maintainers. I would vote to boost to 2.52 or above. Skip From mal@lemburg.com Thu Apr 4 16:37:21 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 04 Apr 2002 18:37:21 +0200 Subject: [Python-Dev] Updating to autoconf 2.5x References: <200204041547.g34Flflv007093@paros.informatik.hu-berlin.de> <15532.31123.535297.863287@12-248-41-177.client.attbi.com> Message-ID: <3CAC8141.83A323C4@lemburg.com> Skip Montanaro wrote: > > Martin> If all regular editors of configure.in have autoconf 2.5x > Martin> available, or can install it, I'd propose to bump the AC_PREREQ > Martin> to 2.50. > > I believe 2.50 had serious bugs and was superceded rather quickly by the > autoconf maintainers. I would vote to boost to 2.52 or above. Does the update buy us anything ? FWIW, Suse 7.2 shipped with autoconf 2.13 and is still a rather recent Linux distro. Side note: I recently found that the socket module is using bleeding edge glibc APIs as well, in fact it was the only module that needed the most recent glibc version installed on my machine (2.2.1). All other modules were happy with plain glibc 2.2. I found out about this when installing an application on RedHat which didn't have glibc 2.2.1 installed. This may be a marginal problem, but do we really need to live on the bleeding edge of C libraries ? And if so, is there a way to configure Python to only use, say, glibc 2.2 APIs (to enhance binary compatibility) ? -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From tim.one@comcast.net Thu Apr 4 16:38:54 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 04 Apr 2002 11:38:54 -0500 Subject: [Python-Dev] Re: Suggested memory API rules for 2.3 In-Reply-To: <20020404063910.A4880@glacier.arctrix.com> Message-ID: [Neil Schemenauer] > People will have to cast to PyObject* when calling PyObject_GC_Del. This depends on whether you want to leave its signature alone, or declare it as taking a void*. > I guess that will be consistent with the other _Free and _Del functions. One reason I prefer the other _Free functions over their _Del versions is that they're already declared to take void*, just like C free(); e.g., extern DL_IMPORT(void) PyObject_Free(void *); Some of our macros do redundant casts to void* now when invoking these guys. In any case, if people stick to the "recommended" API {PyMem, PyObject)_Free, they should not need to cast their arguments. Note that there's no type safety in the way we've actually implemented things even for the PyObject_Del spelling: #define PyObject_Del(op) _PyObject_Del((PyObject *)(op)) That is, no matter what kind of goofy pointer a programmer may pass, we silently cast it to PyObject* anyway. Better then (IMO) to change _PyObject_Del's signature to void*, let *it* cast to PyObject* internally, and lose the macro trick. From guido@python.org Thu Apr 4 16:47:47 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 04 Apr 2002 11:47:47 -0500 Subject: [Python-Dev] Updating to autoconf 2.5x In-Reply-To: Your message of "Thu, 04 Apr 2002 17:47:41 +0200." <200204041547.g34Flflv007093@paros.informatik.hu-berlin.de> References: <200204041547.g34Flflv007093@paros.informatik.hu-berlin.de> Message-ID: <200204041647.g34GllA01982@odiug.zope.com> > Folks, > > autoconf maintainers have advanced autoconf development over the last > few years in slightly incompatible ways; as a result, the have > produced autoconf 2.50 and successors. > > The generated autoconf scripts vary largely in their text, and > slightly in their behaviour; autoconf 2.50 offers many additional > features, also, not all configure.in scripts work with autoconf 2.50. > > Access to autoconf 2.13 with Linux distributions is becoming a > problem, since some Linux distributors only offer 2.5x packages. So > the question arises whether Python's configure should update to 2.50. > > In rev 1.268, I had ported configure.in to support autoconf 2.50, but > have continued to use autoconf 2.13 to regenerate it. > > If all regular editors of configure.in have autoconf 2.5x available, > or can install it, I'd propose to bump the AC_PREREQ to 2.50. > > For those of you who'd then need both autoconf versions available, I > can recommend the Debian wrapper; this is a perl script that invokes > either autoconf2.13 or autoconf2.50, depending on several criteria > (configure.in or configure.ac? AC_PREREQ?). If there is interest, I > can post instructions on how to install both versions side-by-side. > > So what do you think? > > Regards, > Martin No problem from me. autoconf 2.50 works perfectly for me. But when I downloaded and installed autoconf 2.53, this is what I got when I ran it: $ autoconf configure.in:1062: error: do not use LIBOBJS directly, use AC_LIBOBJ (see section `AC_LIBOBJ vs. LIBOBJS' Autoheader gives me a bunch of new warnings: $ autoheader WARNING: Using auxiliary files such as `acconfig.h', `config.h.bot' WARNING: and `config.h.top', to define templates for `config.h.in' WARNING: is deprecated and discouraged. WARNING: Using the third argument of `AC_DEFINE' and WARNING: `AC_DEFINE_UNQUOTED' allows to define a template without WARNING: `acconfig.h': WARNING: AC_DEFINE([NEED_MAIN], 1, WARNING: [Define if a function `main' is needed.]) WARNING: More sophisticated templates can also be produced, see the WARNING: documentation. autoheader: `pyconfig.h.in' is updated Should we stick with 2.50, or make configure.in compatible with 2.53? I do think that we should agree on using the same version of autoconf, to prevent huge massive changes to the configure script each time someone with a different version regenerates it. --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Thu Apr 4 16:52:37 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 4 Apr 2002 11:52:37 -0500 Subject: [Python-Dev] Updating to autoconf 2.5x References: <200204041547.g34Flflv007093@paros.informatik.hu-berlin.de> <15532.31123.535297.863287@12-248-41-177.client.attbi.com> <3CAC8141.83A323C4@lemburg.com> Message-ID: <15532.34005.146558.767800@anthem.wooz.org> >>>>> "MAL" == M writes: MAL> FWIW, Suse 7.2 shipped with autoconf 2.13 and is still a MAL> rather recent Linux distro. Mandrake 8.2, shipped just a few weeks ago, also has autoconf 2.13 as the default. -Barry From tim.one@comcast.net Thu Apr 4 16:56:03 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 04 Apr 2002 11:56:03 -0500 Subject: [Python-Dev] Problem with the memory docs In-Reply-To: <200204041454.g34Es8210623@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Tim] >> The docs for PyMem_Malloc say: >> >> Requesting zero bytes returns a non-NULL pointer. >> ... [Guido] > The whole post is a joke, right? No -- I can't implement what the docs promise there. > You just mean that it can still return NULL when it runs out of memory, > so the docs should say that? The docs should be accurate, yes. From skip@pobox.com Thu Apr 4 16:59:41 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 4 Apr 2002 10:59:41 -0600 Subject: [Python-Dev] Updating to autoconf 2.5x In-Reply-To: <3CAC8141.83A323C4@lemburg.com> References: <200204041547.g34Flflv007093@paros.informatik.hu-berlin.de> <15532.31123.535297.863287@12-248-41-177.client.attbi.com> <3CAC8141.83A323C4@lemburg.com> Message-ID: <15532.34429.348610.709426@12-248-41-177.client.attbi.com> >>>>> "mal" == mal writes: mal> Skip Montanaro wrote: >> Martin> If all regular editors of configure.in have autoconf 2.5x Martin> available, or can install it, I'd propose to bump the AC_PREREQ Martin> to 2.50. >> >> I believe 2.50 had serious bugs and was superceded rather quickly by the >> autoconf maintainers. I would vote to boost to 2.52 or above. mal> Does the update buy us anything ? Maybe not. When I was making daily builds of Gtk2 from CVS a few months ago, I needed to install autoconf 2.52. The Gtk folks use features not available in 2.13. Martin thinks there are some bug fixes. The biggest difference I see is that when I make a change to configure.in, rerun autoconf, then want to see what the resulting change to configure is, ediff reports 1355 changes instead of a small handful. If we have everyone on different versions of autoconf, configure script checkins are going to be humongous. I doubt we'll fill up the CVS disk on SF, but it still seems like we should move forward as a group when we do, particularly those folks who edit configure.in most frequently. I can't comment on glibc versions. I just go with the flow there... Skip From kbutler@campuspipeline.com Thu Apr 4 17:08:22 2002 From: kbutler@campuspipeline.com (Kevin Butler) Date: Thu, 04 Apr 2002 10:08:22 -0700 Subject: [Python-Dev] ANN: Pyrex - a language for writing Python extension modules Message-ID: <3CAC8886.9050302@campuspipeline.com> Greg Ewing wrote: > Pyrex is more or less Python with C data types > added. You can write functions which freely > intermix operations on Python and C data, with > all Python reference counting and error checking > completely automated. > > An example Pyrex module is shown below. For more > information, see: > > http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/ Greg, this is beautiful. Exactly what I've been wanting. :-) Comments/suggestions: 1- Any chance of having "typecast" spelled as one of: char*(value) #consistent w/ Python (char *)value #consistent w/ C cdef char*(value) #cdef + Python (cdef char *)value #cdef + C cdef(char*, value) #umm. Maybe easier for parsing? I prefer any of the above over the <> syntax, in rough order of preference... 2- I like the use of the explicit extern declarations to "import" external C items. It is analogous to "from X import NAME", rather than the "from X import *" behavior suggested by parsing header files. Also, from previous experience w/ SWIG, et al, I think parsing C (& especially C++!) headers is likely to bring in more headaches than it is worth. How deeply do you want to get into macros, typedefs, other extern declarations, etc.? 3- Any thoughts on accessing C++ classes? It need only include the info to access the subset of members that are needed in Python/Pyrex... Maybe something like: cdef extern class Grail: # all members implicity "cdef extern" int age float volume # To self or not to self? # Let declarations be consistent w/ C++? def __init__( int age, float volume ) def float quaff( float amount ) Wow! Look at those worms wiggle out of that can! ;-) (Next, I'll hit you up for exceptions and default args, then overloading, then templates & RTTI... ) kb From guido@python.org Thu Apr 4 17:27:18 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 04 Apr 2002 12:27:18 -0500 Subject: [Python-Dev] Updating to autoconf 2.5x In-Reply-To: Your message of "Thu, 04 Apr 2002 18:37:21 +0200." <3CAC8141.83A323C4@lemburg.com> References: <200204041547.g34Flflv007093@paros.informatik.hu-berlin.de> <15532.31123.535297.863287@12-248-41-177.client.attbi.com> <3CAC8141.83A323C4@lemburg.com> Message-ID: <200204041727.g34HRIR05506@odiug.zope.com> > Side note: I recently found that the socket module is using > bleeding edge glibc APIs as well, in fact it was the only > module that needed the most recent glibc version installed > on my machine (2.2.1). Which API? > All other modules were happy with plain glibc 2.2. I found > out about this when installing an application on RedHat which > didn't have glibc 2.2.1 installed. > > This may be a marginal problem, but do we really need to > live on the bleeding edge of C libraries ? And if so, is > there a way to configure Python to only use, say, glibc 2.2 > APIs (to enhance binary compatibility) ? If we're using bleeding edge APIs, they should be properly tested for in configure.in and the code should use #ifdef. --Guido van Rossum (home page: http://www.python.org/~guido/) From Gerson.Kurz@t-online.de Thu Apr 4 18:29:01 2002 From: Gerson.Kurz@t-online.de (Gerson Kurz) Date: Thu, 4 Apr 2002 19:29:01 +0100 Subject: AW: [Python-Dev] Python sidebar for Mozilla/Netscape In-Reply-To: <15532.28675.583270.786889@12-248-41-177.client.attbi.com> Message-ID: Skip Montanaro wrote > I saw all the other responses so far, and while it looks cute, I can only > think of two reasons to use Mozilla/NS6: testing for compatibility and > escape from MS. Writing code that is specific to any browser seems > counterproductive to me. I know, its a blatant self-advertisement, but maybe you want to take a look at http://www.p-nand-q.com/cgi-bin/cgidocs.py? Runs with any browser than can handle frames. Source is here: http://www.p-nand-q.com/cgidocs.htm. Could be easily changed to omit mysql dependency. I have a version that allows searching the docs by utilizing google.com, but unfortunately google license prohibits querying google from a program :( From martin@v.loewis.de Thu Apr 4 18:26:55 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 04 Apr 2002 20:26:55 +0200 Subject: [Python-Dev] New canonical location for PEPs In-Reply-To: <4rishsav.fsf@smtprelay.t-online.de> References: <15530.31727.557290.92540@grendel.zope.com> <20020402233722.H16962@tummy.com> <20020403004949.I16962@tummy.com> <87ofh0xo93.fsf@tleepslib.sk.tsukuba.ac.jp> <4rishsav.fsf@smtprelay.t-online.de> Message-ID: Adrian Aichner writes: > See the XEmacs/xemacsweb entry in > http://cvs.xemacs.org/viewcvs.cgi/CVSROOT/loginfo?rev=HEAD&content-type=text/vnd.viewcvs-markup Thanks for this information. I'm still not sure what the status of automated remote update is in XEmacs at the moment, sometimes, it says that it's disabled because it doesn't work; various pieces still seem to be in place. More specifically, it appears that CVSROOT/commit-trigger is the driver that initiates all the remote updates. I see it does that for xemacs@sunsite.dk and two other systems. What I cannot see is what programs are run at the remote system. Are those available somewhere? Every time I tried to setup such a scheme, I found that it would deadlock: while the loginfo code runs, the committer still holds the lock. Then, the trigger would initiate code on the remote machine, which in turn tries a CVS update. That will then wait for the lock. How do you solve this problem? Regards, Martin From martin@v.loewis.de Thu Apr 4 18:47:07 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 04 Apr 2002 20:47:07 +0200 Subject: [Python-Dev] ACCEPTED: PEP 285 In-Reply-To: <3CAB8C99.232A8571@metaslash.com> References: <200204032232.g33MWqJ28933@odiug.zope.com> <3CAB8C99.232A8571@metaslash.com> Message-ID: Neal Norwitz writes: > def test(a): > if a is True: print 'True' > if a == False: print 'False' > > generates these warnings: > test.py:2: Should not assign to True, it is (or will be) a builtin > test.py:3: Should not assign to False, it is (or will be) a builtin While Gustavo suggests to let pass 'a is True', I recommend to give a warning for this - people who want to see whether something is True should write 'True is a'. Regards, Martin From guido@python.org Thu Apr 4 18:59:16 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 04 Apr 2002 13:59:16 -0500 Subject: [Python-Dev] ACCEPTED: PEP 285 In-Reply-To: Your message of "04 Apr 2002 20:47:07 +0200." References: <200204032232.g33MWqJ28933@odiug.zope.com> <3CAB8C99.232A8571@metaslash.com> Message-ID: <200204041859.g34IxGF07438@odiug.zope.com> > While Gustavo suggests to let pass 'a is True', I recommend to give a > warning for this - people who want to see whether something is True > should write 'True is a'. Yuck. --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Thu Apr 4 19:06:44 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 4 Apr 2002 14:06:44 -0500 Subject: [Python-Dev] New canonical location for PEPs References: <15530.31727.557290.92540@grendel.zope.com> <20020402233722.H16962@tummy.com> <20020403004949.I16962@tummy.com> <87ofh0xo93.fsf@tleepslib.sk.tsukuba.ac.jp> <4rishsav.fsf@smtprelay.t-online.de> Message-ID: <15532.42052.482154.740180@anthem.wooz.org> >>>>> "MvL" == Martin v Loewis writes: MvL> Every time I tried to setup such a scheme, I found that it MvL> would deadlock: while the loginfo code runs, the committer MvL> still holds the lock. Then, the trigger would initiate code MvL> on the remote machine, which in turn tries a CVS update. That MvL> will then wait for the lock. MvL> How do you solve this problem? syncmail has the same problem; it solves it by forking() a child process which waits a second or two. Sucks, but I know of no other way. -Barry From barry@zope.com Thu Apr 4 19:08:29 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 4 Apr 2002 14:08:29 -0500 Subject: [Python-Dev] ACCEPTED: PEP 285 References: <200204032232.g33MWqJ28933@odiug.zope.com> <3CAB8C99.232A8571@metaslash.com> Message-ID: <15532.42157.607160.633720@anthem.wooz.org> >>>>> "MvL" == Martin v Loewis writes: MvL> While Gustavo suggests to let pass 'a is True', I recommend MvL> to give a warning for this - people who want to see whether MvL> something is True should write 'True is a'. "True is a"... what? hippopotamus? . -Barry From martin@v.loewis.de Thu Apr 4 18:28:39 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 04 Apr 2002 20:28:39 +0200 Subject: [Python-Dev] Deprecating C APIs (Use of PyArg_NoArgs()) In-Reply-To: <03ca01c1daff$b5b3f530$ced241d5@hagrid> References: <3C9F4AC0.C076679B@metaslash.com> <3C9F8B0F.D921E530@metaslash.com> <200203252052.g2PKqJa08509@pcp742651pcs.reston01.va.comcast.net> <3C9FA9D1.DA46BE04@metaslash.com> <200203252300.g2PN0DY09544@pcp742651pcs.reston01.va.comcast.net> <3CA9777F.BB3DA0DD@lemburg.com> <200204021437.g32Eb7C09097@pcp742651pcs.reston01.va.comcast.net> <3CAA0290.24D1E302@lemburg.com> <200204022104.g32L4pC20305@odiug.zope.com> <03ca01c1daff$b5b3f530$ced241d5@hagrid> Message-ID: "Fredrik Lundh" writes: > and what good is that, really? why would the *users* of a > module I've written have to care about this? > > the Py_DEPRECATED approach (and the "grep" tool) is a much > better way to use everybody's time. Your users will care about it because your module fails to compile if the deprecated API is wrapped with Py_DEPRECATED, whereas it continues to compile and run when the warning is produced - the warning is annoying (and really meant for you only), but is the lesser evil. Regards, Martin From jamescalthoff@yahoo.com Thu Apr 4 19:25:56 2002 From: jamescalthoff@yahoo.com (James Althoff) Date: Thu, 4 Apr 2002 11:25:56 -0800 Subject: [Python-Dev] Re: PEP 279 Message-ID: <000801c1dc0e$8c23e4e0$54dac13f@Pacbell.net> This is a multi-part message in MIME format. ------=_NextPart_000_0005_01C1DBCB.7D512B00 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable [David Abrahams] > . . . I want to put in another plug for itemize(), Well, since we're doing post-Pronouncement fantasizing . . . here's one more plug for "itemize". Noting that for k,v in adict.iteritems(): is probably as useful as for i,v in enumerate(aseq): why not change the spec to: >>> def itemize(iterable): ... # use iteritems if defined ... try: ... for item in iterable.iteritems(): yield item ... except AttributeError: pass ... # and then everything as before ... i =3D 0 ... iterator =3D iter(iterable) ... while 1: ... yield i,iterator.next() ... i +=3D 1 ... So "itemize(iterable)" returns an iterator that yields (k,v) pairs from = the iterable.iteritems() method if defined, else pairs generated by = associating 0,1,...,n-1 with n values from iter(iterable). This allows for k,v in itemize(adict_or_aseq): to be written uniformly for dicts and seqs. And makes=20 list(itemize(adict)) =3D=3D adict.items() work, removing one of the objections to the name "itemize". Also, if one were to need, for example, a sequence whose indices start = at 1 instead of 0, one could define a sequence class that implements = iteritems and objects of said class would work just fine with "itemize". Jim ------=_NextPart_000_0005_01C1DBCB.7D512B00 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
[David Abrahams]
> . . . I want to put in = another plug for=20 itemize(),
Well, since we're doing = post-Pronouncement=20 fantasizing . . .
 
here's one more plug for=20 "itemize".
 
Noting that
 
    for k,v in=20 adict.iteritems():
 
is probably as useful as
 
    for i,v in=20 enumerate(aseq):
 
why not change=20 the spec to:
 
>>> def=20 itemize(iterable):
...     # use iteritems if=20 defined
...    =20 try:
...         for item in=20 iterable.iteritems(): yield item
...     except=20 AttributeError: pass
...     # and then = everything as=20 before
...     i =3D = 0
...    =20 iterator =3D iter(iterable)
...     while=20 1:
...         yield=20 i,iterator.next()
...         = i +=3D=20 1
...
So = "itemize(iterable)" returns an=20 iterator that yields (k,v) pairs from the iterable.iteritems() method if = defined, else pairs generated by associating 0,1,...,n-1 with n values = from=20 iter(iterable).
 
This=20 allows
 
    = for k,v in=20 itemize(adict_or_aseq):
 
to be written = uniformly for dicts=20 and seqs.
 
And makes
 
    = list(itemize(adict)) =3D=3D=20 adict.items()
 
work, removing one of the objections to = the name=20 "itemize".
 
Also, if one were to = need, for=20 example, a sequence whose indices start at 1 instead of 0, one = could define=20 a  sequence class that implements iteritems and objects of said = class would=20 work just fine with = "itemize".
 
Jim
------=_NextPart_000_0005_01C1DBCB.7D512B00-- From jamescalthoff@yahoo.com Thu Apr 4 19:33:13 2002 From: jamescalthoff@yahoo.com (James Althoff) Date: Thu, 4 Apr 2002 11:33:13 -0800 Subject: [Python-Dev] Re: PEP 279 Message-ID: <000701c1dc0f$90475a60$54dac13f@Pacbell.net> Trying again without all the nasty HTML -- apologies to all. [David Abrahams] > . . . I want to put in another plug for itemize(), Well, since we're doing post-Pronouncement fantasizing . . . here's one more plug for "itemize". Noting that for k,v in adict.iteritems(): is probably as useful as for i,v in enumerate(aseq): why not change the spec to: >>> def itemize(iterable): ... # use iteritems if defined ... try: ... for item in iterable.iteritems(): yield item ... except AttributeError: pass ... # and then everything as before ... i = 0 ... iterator = iter(iterable) ... while 1: ... yield i,iterator.next() ... i += 1 ... So "itemize(iterable)" returns an iterator that yields (k,v) pairs from the iterable.iteritems() method if defined, else pairs generated by associating 0,1,...,n-1 with n values from iter(iterable). This allows for k,v in itemize(adict_or_aseq): to be written uniformly for dicts and seqs. And makes list(itemize(adict)) == adict.items() work, removing one of the objections to the name "itemize". Also, if one were to need, for example, a sequence whose indices start at 1 instead of 0, one could define a sequence class that implements iteritems and objects of said class would work just fine with "itemize". Jim From fredrik@pythonware.com Thu Apr 4 17:19:46 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 4 Apr 2002 19:19:46 +0200 Subject: [Python-Dev] Problem with the memory docs References: <200204041454.g34Es8210623@pcp742651pcs.reston01.va.comcast.net> Message-ID: <028201c1dc0f$4966f970$ced241d5@hagrid> tim wrote: > > Requesting zero bytes returns a non-NULL pointer. > > I can't guarantee this without insane effort, and our code has never > guaranteed this. umm. I have to admit that it's rather ugly, but I'm pretty sure the following guarantees that requesting zero bytes will never return a NULL pointer: static char nullbyte = MAGIC; void* malloc(int bytes) { if (bytes == 0) { #if DEBUG_MODE if (*nullbyte != MAGIC) uh-oh #endif return &nullbyte; } ... void free(void* ptr) { if (ptr == &nullbyte) { #if DEBUG_MODE if (*nullbyte != MAGIC) uh-oh #endif return; /* nothing to do */ } ... From martin@v.loewis.de Thu Apr 4 19:34:46 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 04 Apr 2002 21:34:46 +0200 Subject: [Python-Dev] Updating to autoconf 2.5x In-Reply-To: <3CAC8141.83A323C4@lemburg.com> References: <200204041547.g34Flflv007093@paros.informatik.hu-berlin.de> <15532.31123.535297.863287@12-248-41-177.client.attbi.com> <3CAC8141.83A323C4@lemburg.com> Message-ID: "M.-A. Lemburg" writes: > Does the update buy us anything ? I know of two specific aspects: - the AC_CHECK_HEADERS in autoconf 2.5x will check the status of the compiler invocation, instead of checking the error output. In turn, if gcc produces a warning for a system header, autoconf 2.13 will conclude that the headers is missing, whereas autoconf 2.5x will detect its presence. This, in turn, should fix #535545. - autoconf 2.5x supports AC_CHECK_DECL. This can help fixing #534108. > FWIW, Suse 7.2 shipped with autoconf 2.13 and is still a rather > recent Linux distro. SuSE 7.3 ships 2.52 only. It is a more recent Linux distribution. > Side note: I recently found that the socket module is using > bleeding edge glibc APIs as well, in fact it was the only > module that needed the most recent glibc version installed > on my machine (2.2.1). Can you give precise details? > This may be a marginal problem, but do we really need to > live on the bleeding edge of C libraries ? And if so, is > there a way to configure Python to only use, say, glibc 2.2 > APIs (to enhance binary compatibility) ? It certainly does; if it doesn't, please report a bug. In any case, Python is not written specifically for glibc; it is portable across various C libraries. Regards, Martin From martin@v.loewis.de Thu Apr 4 19:35:28 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 04 Apr 2002 21:35:28 +0200 Subject: [Python-Dev] Updating to autoconf 2.5x In-Reply-To: <200204041647.g34GllA01982@odiug.zope.com> References: <200204041547.g34Flflv007093@paros.informatik.hu-berlin.de> <200204041647.g34GllA01982@odiug.zope.com> Message-ID: Guido van Rossum writes: > Should we stick with 2.50, or make configure.in compatible with 2.53? The latter. I'll look into this. Regards, Martin From martin@v.loewis.de Thu Apr 4 19:54:52 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 04 Apr 2002 21:54:52 +0200 Subject: [Python-Dev] Problem with the memory docs In-Reply-To: <028201c1dc0f$4966f970$ced241d5@hagrid> References: <200204041454.g34Es8210623@pcp742651pcs.reston01.va.comcast.net> <028201c1dc0f$4966f970$ced241d5@hagrid> Message-ID: "Fredrik Lundh" writes: > umm. I have to admit that it's rather ugly, but I'm pretty sure the > following guarantees that requesting zero bytes will never return a > NULL pointer: As Tim explained earlier, this implementation violates the ISO C requirement that different calls of malloc return different pointers. Regards, Martin From fredrik@pythonware.com Thu Apr 4 19:56:46 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 4 Apr 2002 21:56:46 +0200 Subject: [Python-Dev] Deprecating C APIs (Use of PyArg_NoArgs()) References: <3C9F4AC0.C076679B@metaslash.com><3C9F8B0F.D921E530@metaslash.com><200203252052.g2PKqJa08509@pcp742651pcs.reston01.va.comcast.net><3C9FA9D1.DA46BE04@metaslash.com><200203252300.g2PN0DY09544@pcp742651pcs.reston01.va.comcast.net><3CA9777F.BB3DA0DD@lemburg.com><200204021437.g32Eb7C09097@pcp742651pcs.reston01.va.comcast.net><3CAA0290.24D1E302@lemburg.com><200204022104.g32L4pC20305@odiug.zope.com><03ca01c1daff$b5b3f530$ced241d5@hagrid> Message-ID: <02c501c1dc13$83897e80$ced241d5@hagrid> Martin v. Loewis wrote: > "Fredrik Lundh" writes: > > > and what good is that, really? why would the *users* of a > > module I've written have to care about this? > > > > the Py_DEPRECATED approach (and the "grep" tool) is a much > > better way to use everybody's time. > > Your users will care about it because your module fails to compile if > the deprecated API is wrapped with Py_DEPRECATED oh, please. 70-80% of my users use libraries that someone else compiled for them. giving meaningless warnings is a great way to make them question the quality of the software (I had enough of that when 2.1 started complaining about shadowed globals), but it won't help them a bit. > whereas it continues to compile and run when the warning is produced >- the warning is annoying (and really meant for you only), but is the > lesser evil. if the message is meant for me, please make sure I see it. the C compiler/linker has no problem doing that. but I cannot guarantee that every library I've ever written has a test suite that tests every piece of the library, or that the test suite is set up to look for warnings (doctest won't catch them, for example). From fredrik@pythonware.com Thu Apr 4 20:01:26 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 4 Apr 2002 22:01:26 +0200 Subject: [Python-Dev] Problem with the memory docs References: <200204041454.g34Es8210623@pcp742651pcs.reston01.va.comcast.net><028201c1dc0f$4966f970$ced241d5@hagrid> Message-ID: <02c801c1dc13$84254950$ced241d5@hagrid> Martin v. Loewis wrote: > > > umm. I have to admit that it's rather ugly, but I'm pretty sure the > > following guarantees that requesting zero bytes will never return a > > NULL pointer: > > As Tim explained earlier, this implementation violates the ISO C > requirement that different calls of malloc return different pointers. oh. last time I checked, the ISO C standard didn't define the behaviour of any Py_ functions... (maybe you meant ISO Python? ;-) (the documentation says "modelled after", but that doesn't necessarily mean "unless otherwise specified, behaves exactly as the corresponding functions in the"...) From pedroni@inf.ethz.ch Thu Apr 4 20:15:35 2002 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Thu, 4 Apr 2002 22:15:35 +0200 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format References: <014301c1dbe5$bbbffdc0$6d94fea9@newmexico> Message-ID: <03b201c1dc15$7b0f2aa0$6d94fea9@newmexico> [me] > The only thing I found relevant in the PEP are > (does this fact means something?): > > > - Markup that isolates a Python identifier: interpreted text. > > - Markup that isolates a Python identifier and specifies its type: > interpreted text with roles. > Ok there's another relevant bit: - Markup for "tagged lists" or "label lists": field lists. ... They may be used to implement docstring semantics, such as identifying parameters, exceptions raised, etc.; such usage is beyond the scope of this PEP. how many eons would pass before we see a PEP about those *relevant* issues? Presumably JavaDoc is a good start point for this . More or less JavaDoc tags would map to fields, except @see and @link that would use `interp-text` and the various ref/links forms. regards. From fredrik@pythonware.com Thu Apr 4 20:28:54 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 4 Apr 2002 22:28:54 +0200 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format References: <014301c1dbe5$bbbffdc0$6d94fea9@newmexico> <03b201c1dc15$7b0f2aa0$6d94fea9@newmexico> Message-ID: <03b301c1dc17$58cfb390$ced241d5@hagrid> samuele wrote: > how many eons would pass before we see a PEP about > those *relevant* issues? Presumably JavaDoc is a good > start point for this . do you need a PEP, or is it okay if I just post some code? From adrian@xemacs.org Thu Apr 4 19:34:51 2002 From: adrian@xemacs.org (Adrian Aichner) Date: 04 Apr 2002 21:34:51 +0200 Subject: [Python-Dev] New canonical location for PEPs In-Reply-To: References: <15530.31727.557290.92540@grendel.zope.com> <20020402233722.H16962@tummy.com> <20020403004949.I16962@tummy.com> <87ofh0xo93.fsf@tleepslib.sk.tsukuba.ac.jp> <4rishsav.fsf@smtprelay.t-online.de> Message-ID: >>>>> "Martin" == Martin v Loewis writes: Martin> Adrian Aichner writes: >> See the XEmacs/xemacsweb entry in >> http://cvs.xemacs.org/viewcvs.cgi/CVSROOT/loginfo?rev=HEAD&content-type=text/vnd.viewcvs-markup Martin> Thanks for this information. I'm still not sure what the Martin> status of automated remote update is in XEmacs at the Martin> moment, sometimes, it says that it's disabled because it Hi Martin, where does it say it's disabled or wouldn't work. Please send me pointers. I'd appreciate it. The xemacsweb module has been updated via the commit-trigger for a long time now. See the XEmacs Community News section of http://www.xemacs.org/ starting with the entry of 2000-11-26. Martin> doesn't work; various pieces still seem to be in place. The most recent addition to the website are manuals generated from .texi using texi2html. This part is not automated yet. Martin> More specifically, it appears that CVSROOT/commit-trigger Martin> is the driver that initiates all the remote updates. I see Martin> it does that for xemacs@sunsite.dk and two other Martin> systems. What I cannot see is what programs are run at the Correct. Martin> remote system. Are those available somewhere? Ah, the information you are missing are the relevant authorized_keys entries. Here is the one from xemacs@www.us.xemacs.org no-port-forwarding,no-X11-forwarding,no-agent-forwarding,environment="PATH=/hom\ e/groups/x/xe/xemacs/tools/xemacs/bin:/usr/bin:/bin",from="*.sourceforge.net,*.\ m17n.org,gwyn.tux.org,www.xemacs.org,sunsite.dk",command="(/bin/date -u; cat; (\ sleep 2; cd /home/groups/x/xe/xemacs/xemacsweb && cvs -f -z3 -d:pserver:cvs@sun\ site.dk:/pack/xemacscvs update -Pd && make all; /bin/date -u) &) >> /home/group\ s/x/xe/xemacs/xemacsweb/logs/`/bin/date -u +%Y-%m-%dT%H-%M-%SZ`.txt 2>&1" XXXX\ YY ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ\ ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ\ ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ\ ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ XE\ macs Commit Trigger Martin> Every time I tried to setup such a scheme, I found that it Martin> would deadlock: while the loginfo code runs, the committer Martin> still holds the lock. Then, the trigger would initiate Martin> code on the remote machine, which in turn tries a CVS Martin> update. That will then wait for the lock. Martin> How do you solve this problem? Try a commit-trigger based on ours in combination with an adjusted authorized_keys entry similar to the one above. Hope this helps, Adrian Martin> Regards, Martin> Martin -- Adrian Aichner mailto:adrian@xemacs.org http://www.xemacs.org/ From tim.one@comcast.net Thu Apr 4 20:48:52 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 04 Apr 2002 15:48:52 -0500 Subject: [Python-Dev] Problem with the memory docs In-Reply-To: <028201c1dc0f$4966f970$ced241d5@hagrid> Message-ID: [/F] > umm. I have to admit that it's rather ugly, but I'm pretty sure the > following guarantees that requesting zero bytes will never return a > NULL pointer: > > static char nullbyte = MAGIC; > > void* malloc(int bytes) > { > if (bytes == 0) { > #if DEBUG_MODE > if (*nullbyte != MAGIC) > uh-oh > #endif > return &nullbyte; > } > ... > > void free(void* ptr) > { > if (ptr == &nullbyte) { > #if DEBUG_MODE > if (*nullbyte != MAGIC) > uh-oh > #endif > return; /* nothing to do */ > } > ... Don't forget realloc too, and that malloc hasn't taken an int argument since before 1989 . Sorry, but this is nuts, and Martin is right that I don't want Python to offer a "malloc wrapper" that doesn't meet the standard rules for malloc. Instead I intend to change the docs to say that Py_Malloc(0) acts like platform malloc(1), and leave it at that. I expect the _PyMem_EXTRA hack to go away too. From tree@basistech.com Thu Apr 4 20:53:12 2002 From: tree@basistech.com (Tom Emerson) Date: Thu, 4 Apr 2002 15:53:12 -0500 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) In-Reply-To: <018e01c1dbee$51b1c9f0$e000a8c0@thomasnotebook> References: <3CAC2F2B.1950AD8F@lemburg.com> <018e01c1dbee$51b1c9f0$e000a8c0@thomasnotebook> Message-ID: <15532.48440.566653.321044@magrathea.basistech.com> Thomas Heller writes: > I'm interested in feedback: Is it usefull at all? Should > something like this go into the Python distribution? How can it be > improved? What should be the user interface - currently it is command > line only and usable from the interactive interpreter programmatically. Cool: I had to modify it slightly to run correctly on my system (FreeBSD 4.4 with Lynx) (change below) but otherwise it's great. [...] > webbrowser.open("file:%s" % path) [...] Lynx doesn't like the URL that is generated, so I changed to webbrowser.open(path) I expect that should work on any platform. -tree -- Tom Emerson Basis Technology Corp. Sr. Computational Linguist http://www.basistech.com "Beware the lollipop of mediocrity: lick it once and you suck forever" From mhammond@skippinet.com.au Thu Apr 4 23:45:03 2002 From: mhammond@skippinet.com.au (Mark Hammond) Date: Fri, 5 Apr 2002 09:45:03 +1000 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape In-Reply-To: <15532.28675.583270.786889@12-248-41-177.client.attbi.com> Message-ID: [Skip] > Mark> I wrote a cute little Python sidebar for Mozilla/NS6. > > I saw all the other responses so far, and while it looks cute, I can only > think of two reasons to use Mozilla/NS6: testing for compatibility and > escape from MS. I don't want to evangelize Mozilla too much, but FWIW I recently switched my default browser from IE to Mozilla. The killer features for me: * Reasonable image and cookie blocking built in - haven't seen many banner ads for some time now, and no need to install extra software for the blocking. * Nice "script security" features. You can remove the ability of pages to create popup windows or to move or resize the Window. I haven't seen a single popup window in *ages* * "Tabbed" browsing - middle-click on a link, and a new tab opens in the current window. Very useful as it allows you to keep many related web pages open in a single frame. * Keywords, as described here recently. I enter "dict foobar" in the URL bar, and "foobar" is looked up in an online dictionary for me. I have a number of such keywords setup. * Sidebars - the "search" sidebar is particularly effective. The Mozilla developer specific sidebars, (bugzilla, tinderbox autobuild, lxr text search etc) are great. This new Python one is pretty cool too:) * Other "Power User" oriented functions - eg "Page Info" shows a whole slew of info about the current page that is very useful when trouble-shooting pages or web applications. I still don't use Mozilla for mail though - still a few rough edges there. I expect to move to mail over the next 6 months or though. I do use it for news. > Writing code that is specific to any browser seems > counterproductive to me. counterproductive for who? The alternative appears to have been "don't do anything". The fact that this only works in Mozilla is still quite productive for me :) Further, I would expect that in 12 months time, we find Mozilla has the largest browser share for this particular little group! Obviously this is off-topic for python-dev, but I thought some of you may be interested in the fact that Mozilla is stabilizing, more advanced preferences are finding their way to the UI, and I find the "power user" features make Mozilla a better browser than IE. Obviously YMMV. In a desperate attempt to bring things back on topic, there is also a Python XPCOM plugin available for Mozilla. Unfortunately, if you want to write Python code that executes on the client, you need to ensure Python and PyXPCOM are installed on the client, so not incredibly useful for general use. Mark. From DavidA@ActiveState.com Fri Apr 5 00:03:43 2002 From: DavidA@ActiveState.com (David Ascher) Date: Thu, 04 Apr 2002 16:03:43 -0800 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape References: Message-ID: <3CACE9DF.7E7F199E@activestate.com> [Skip] > Mark> I wrote a cute little Python sidebar for Mozilla/NS6. > > I saw all the other responses so far, and while it looks cute, I can only > think of two reasons to use Mozilla/NS6: testing for compatibility and > escape from MS. That's probably due to a misunderstanding. The Google toolbar for IE is darned useful, even if it's only for IE. It's like saying that writing platform-specific applications is cute and everything should be based on posix only =). Mark's sidebar is not a server feature, it's a client add-on. I agree that server features which require a particular browser are of limited interest if you don't have a particular audience in mind who has that browser. OnTopic: If a brave and generous soul set up lxr, bugzilla, and tinderbox for Python, I expect we'd find more Python users using Mozilla =). --da From pedroni@inf.ethz.ch Fri Apr 5 00:22:01 2002 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Fri, 5 Apr 2002 02:22:01 +0200 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format References: <014301c1dbe5$bbbffdc0$6d94fea9@newmexico> <03b201c1dc15$7b0f2aa0$6d94fea9@newmexico> <03b301c1dc17$58cfb390$ced241d5@hagrid> Message-ID: <04d301c1dc37$e8a194a0$6d94fea9@newmexico> From: Fredrik Lundh > samuele wrote: > > > how many eons would pass before we see a PEP about > > those *relevant* issues? Presumably JavaDoc is a good > > start point for this . > > do you need a PEP, or is it okay if I just post some code? > Code please, see code is less controversial than PEPs... [From PEP 287] The lack of a standard syntax for docstrings has hampered the development of standard tools for extracting docstrings and transforming them into documentation in standard formats (e.g., HTML, DocBook, TeX). There have been a number of proposed markup formats and variations, and many tools tied to these proposals, but without a standard docstring format they have failed to gain a strong following and/or floundered half-finished. Honestly, is that true? JavaDoc would be useful even if it didn't allow HTML but just the @tags, IOW rich formatting is a very secondary issue, indeed programmers are lazy and the important part of JavaDoc is getting proper inter-references and bind the information that can be extracted from code (parms) with their doc description. OTOH Perl POD is used for the standard doc for Perl modules. The API doc that comes from Sun is html produced out of the JavaDoc markup. Python standard doc should be specified in the format described at http://www.python.org/doc/2.2p1/doc/doc.html, let's refer to it as py-tex-markup how does the PEP relate to producing from inline docstrings at least a first cut in a subset of the py-tex-markup format? indeed py-tex-markup is maybe overkill for the casual module, but the issue is also that there is not a tool that produce it from the docs inside a casual module (or there is such tool???), and on the same line of reasoning reST is also maybe overkill especially if it does translate to py-tex-markup considering-every-module-dreams-to-become-a-standard-module-and- -receive-BDFL-blessing-ly y'rs From jafo-python-dev@tummy.com Fri Apr 5 00:47:52 2002 From: jafo-python-dev@tummy.com (Sean Reifschneider) Date: Thu, 4 Apr 2002 17:47:52 -0700 Subject: [Python-Dev] [development doc updates] In-Reply-To: <20020404225927.530B318EAD1@grendel.zope.com>; from fdrake@acm.org on Thu, Apr 04, 2002 at 05:59:27PM -0500 References: <20020404225927.530B318EAD1@grendel.zope.com> Message-ID: <20020404174752.M16962@tummy.com> On Thu, Apr 04, 2002 at 05:59:27PM -0500, Fred L. Drake wrote: >The development version of the documentation has been updated: On the topic of dev documentation... What happened to the info distribution? I had info and HTML in the 2.2 RPM, but only the HTML were around for 2.2.1c2 when I tried to pick it up the other night... Is there going to be an info release for the 2.2.1 final, or should I just leave it as HTML-only? Sean -- Tragedy is when I cut my finger. Comedy is when you fall into an open sewer and die. -- Mel Brooks Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From fdrake@acm.org Fri Apr 5 01:07:31 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 4 Apr 2002 20:07:31 -0500 Subject: [Python-Dev] [development doc updates] In-Reply-To: <20020404174752.M16962@tummy.com> References: <20020404225927.530B318EAD1@grendel.zope.com> <20020404174752.M16962@tummy.com> Message-ID: <15532.63699.122156.945015@grendel.zope.com> [Let's take this to the Doc-SIG; all followups there, please!] Sean Reifschneider writes: > On the topic of dev documentation... What happened to the info > distribution? I had info and HTML in the 2.2 RPM, but only the HTML were > around for 2.2.1c2 when I tried to pick it up the other night... Is there > going to be an info release for the 2.2.1 final, or should I just leave it > as HTML-only? The GNU info files have been contributed, but a patch has just been submitted that I'll try to get to in time. In the past, we've run into problems with elisp library incompatibilies between emacs and xemacs. We'll see how the proposed patch holds up. Even if we aren't able to get it working, someone will probably contribute a version. This is how all recent GNU info versions have been prepared (like for the past four years or so). -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From zen@shangri-la.dropbear.id.au Fri Apr 5 01:46:13 2002 From: zen@shangri-la.dropbear.id.au (Stuart Bishop) Date: Fri, 5 Apr 2002 11:46:13 +1000 Subject: [Python-Dev] PEP 285: Adding a bool type In-Reply-To: <200204041506.g34F6pT10705@pcp742651pcs.reston01.va.comcast.net> Message-ID: On Friday, April 5, 2002, at 01:06 AM, Guido van Rossum wrote: >> On Thursday, April 4, 2002, at 02:22 PM, Stuart Bishop wrote: >> >>> On a side note, writing 'i or False and True' performs much better I mean (i and True) or (i or False) >>> than bool(i) in the test implementation, and flow control statements >>> involving ints are perform much better than flow control using bool. >>> Will there be a performance degredation in the final version, or >>> can this be avoided? >> >> I fixed my test to not do a global lookup on True and flow control >> is no longer slower. > > Do I take it that you were benchmarking with the Python class shown in > the PEP? No - I was benchmarking with the patch posted on Sourceforge (before PEP285 was checked into CVS), patched into a CVS snapshot taken at the same time. Casting to bool using (x and True) or (x or False) : 1.61 Casting to bool using (bool) : 2.68 Flow control on explicit True : 1.42 Flow control on explicit 1 : 0.99 Flow control on variable bound to True : 1.02 Flow control on variable bound to 1 : 0.99 import time count = 1000000 def test1(): ''' Casting to bool using (x and True) or (x or False) ''' nums = xrange(count) start = time.clock() for i in nums: x = (i and True) or (i or False) end = time.clock() return end - start def test2(): ''' Casting to bool using (bool) ''' nums = xrange(count) start = time.clock() for i in nums: x = bool(i) end = time.clock() return end - start def test3(): ''' Flow control on explicit True ''' nums = xrange(count) start = time.clock() for i in nums: if True: pass else: pass end = time.clock() return end - start def test4(): ''' Flow control on explicit 1 ''' nums = xrange(count) start = time.clock() for i in nums: if 1: pass else: pass end = time.clock() return end - start def test5(a = True): ''' Flow control on variable bound to True ''' nums = xrange(count) start = time.clock() for i in nums: if a: pass else: pass end = time.clock() return end - start def test6(a = 1): ''' Flow control on variable bound to 1 ''' nums = xrange(count) start = time.clock() for i in nums: if a: pass else: pass end = time.clock() return end - start for f in (test1,test2,test3,test4,test5,test6): print f.__doc__,': ',f() -- Stuart Bishop http://shangri-la.dropbear.id.au/ From tim.one@comcast.net Fri Apr 5 02:26:31 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 04 Apr 2002 21:26:31 -0500 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape In-Reply-To: Message-ID: [Mark Hammond] > I don't want to evangelize Mozilla too much, but FWIW I recently > switched my default browser from IE to Mozilla. Dear Lord! When I heard you were starting to use Linux several months ago, I was afraid something like this might happen. Luckily, I contacted Bill Gates at your first sign of illness, and you can expect a loving intervention soon. It would have happened earlier, but when your government found out just how much Uncle Bill loved you, they raised the price on Australia. they'll-pry-ie6-from-my-cold-dead-fingers-ly y'rs - tim From tim.one@comcast.net Fri Apr 5 03:14:19 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 04 Apr 2002 22:14:19 -0500 Subject: [Python-Dev] pymalloc killer In-Reply-To: <464156E0633B154091731247D911C9CF0E0CF7@ge0057excuag01.ags.agere.com> Message-ID: >> i.e., all references to pool->capacity *could* be replaced by >> capacity[pool->szidx], where "capacity" is a new small (at most >> 32 uints) file-static const vector computed at compile-time. [Marangozov, Vladimir (Vladimir) -- I hope you're noticing that your name as reported by your email headers is getting stranger all the time ] > The comparison in the "try to extend the free list" code is executed > very frequently, so probing a static vector this way will slow down the > thing noticeably. I've tried that. We are very much running after speed > here. Gotcha. I was left partly unhappy with the "bulletproof address check" for the same reason (needing to index into a static vector of arena base addresses). > ... [on "the division" and "the multiplication" optimizations] > Do both, despite that the multiplication optimization is more important. > (for one division we have #blocks multiplications for a full pool) Two things have shifted since you wrote this code: + The architecture trend has been to make division ever-slower in comparison to multiplication (e.g., Itanium doesn't even have an integer division instruction); on some boxes integer division is over 100 cycles now, at the same time integer multiplication is down to a few. + We can fit fewer objects into a pool now. For example, on Windows, Python container types are 16 bytes bigger than they used to be, due to gc-header overhead (and an empty dict is actually up to 148 bytes on Windows now!). We could shift that back by making pools larger, but if they exceed the size of a system page then cutting a legit pointer back to a pool-aligned address may yield an invalid address. So it gets dicey short of teaching Python how to ask the OS directly for pages. > 1) I think that adding back one uint slot (lastoffset) is not a problem > (and we end up with the perfect balance of having a pool header layout > with 4 pointers and 4 integers ). Yes, that is perfect . > 2) Now, with the advent of lastoffset, we can easily get rid off the > division as well. For this, we switch to offset calculus for figuring > out whether the free list can be extended. This means that 'capacity' > becomes 'highoffset'. The 'highoffset' is the highwater not to be > exceeded by more than 'size' bytes by 'lastoffest'. > > a) the test becomes then: if (pool->lastoffset <= pool->highoffset) > ... > pool->lastoffset += size; > pool->freeblock = (block *)pool + \ > pool->lastoffset; > > b) in the pool_init code: pool->lastoffset = POOL_OVERHEAD + size; > pool->highoffset = POOL_SIZE - size - size; > > and you'll need to adjust the code accordingly at the places where > you're using 'capacity' per se (and adapt the comments, of course). I'll do this, but I expect I'll use a nextoffset instead of a lastoffset, in order to let the two additions in 2a proceed in parallel, and to skip one of the subtracts in 2b. Hey, if we're gonna optimize, I'm not gonna miss a cycle. > This should pretty much settle these murky optimizations for now. Cool! Thanks for your help. You should come back to Python, you know -- we don't know what you're doing instead, but we all have to suspect it's comparatively worthless . From skip@pobox.com Fri Apr 5 03:32:30 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 4 Apr 2002 21:32:30 -0600 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape In-Reply-To: <3CACE9DF.7E7F199E@activestate.com> References: <3CACE9DF.7E7F199E@activestate.com> Message-ID: <15533.6862.404580.343659@12-248-41-177.client.attbi.com> >> I saw all the other responses so far, and while it looks cute, I can >> only think of two reasons to use Mozilla/NS6: testing for >> compatibility and escape from MS. David> Mark's sidebar is not a server feature, it's a client add-on. I David> agree that server features which require a particular browser are David> of limited interest if you don't have a particular audience in David> mind who has that browser. My last words on the subject. I realize it's not really fodder for this list. I don't think it really matters if the neat trick is server-side or client-side. By using such stuff you encourage the browser authors to create more non-standard ways of doing things. Many of those things they create do wind up on the server side, so they do reduce the "portability" (probably not the correct term) of the web. I just love it when I visit a website and it tells me that my browser is "too old" for its advanced features (Opera 6.0 beta 1)... Skip From skip@pobox.com Fri Apr 5 03:47:57 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 4 Apr 2002 21:47:57 -0600 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape In-Reply-To: References: <15532.28675.583270.786889@12-248-41-177.client.attbi.com> Message-ID: <15533.7789.133019.255360@12-248-41-177.client.attbi.com> >> Writing code that is specific to any browser seems counterproductive >> to me. Mark> counterproductive for who? Sorry, I need to respond, only to correct myself. After posting, I realized "counterproductive" was probably also the wrong choice of word. I guess I meant "counterproductive" in the sense of encouraging movement away from proprietary software. If you use their little bells and whistles, you just encourage them to make more. Some, though not all, will help lock you into their software to the exclusion of other otherwise viable alternatives. my-lips-are-now-officially-sealed-zzzzzzziiippp-ly, y'rs, Skip From barry@zope.com Fri Apr 5 04:28:14 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 4 Apr 2002 23:28:14 -0500 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape References: <3CACE9DF.7E7F199E@activestate.com> <15533.6862.404580.343659@12-248-41-177.client.attbi.com> Message-ID: <15533.10206.635597.723733@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: SM> I just love it when I visit a website and it tells me that my SM> browser is "too old" for its advanced features (Opera 6.0 beta SM> 1)... When you're bored, try proxying through junkbuster, which lies about your browser id. Shut off cookies, Java, and Javascript. Then hunt around for sites that don't work. Email their webmasters and tell them that because of your disability you're forced to use a simplified, non-standard browser, and if they want to comply with the ADA they'd better make their sites work for folks like you. Don't tell them that your disability is that you're a bass player . -Barry From goodger@users.sourceforge.net Fri Apr 5 04:33:31 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Thu, 04 Apr 2002 23:33:31 -0500 Subject: [Python-Dev] Re: PEP 287: reStructuredText Standard Docstring Format In-Reply-To: References: <200204042338.XAA12847@crown.off.ekorp.com> <01c701c1dc35$5a2312f0$02010a0a@suxlap> Message-ID: Thank you to everyone who has commented on PEP 287, be they in favor, well-meaning but opposed, or ill-wishing troll ;-) [1]_. I have been otherwise busy and haven't been able to keep up with replies, but I have read and kept a record of all the posts so far (and will, for posts yet to come) for consideration and reference when revising the PEP. I may not be able to offer timely replies to all the posts; ultimately the revised PEP will serve as an umbrella reply. .. [1] Actually, the posts opposed and even the trolls are often the most useful, since they point out weaknesses and help strengthen the arguments. The number of people posting *in favor* of the PEP was roughly the same as the number opposed, but supporters tend to be silent while protesters tend to be quite vociferous and relentless. Over the weeks to come, I will be revising PEP 287 and related PEPs 256-258, merging the reStructuredText_ and DPS_ projects into the renamed Docutils_ project, and producing a "road map" describing the big picture. Further suggestions for improvement are welcome. .. _reStructuredText: http://structuredtext.sourceforge.net/ .. _DPS: http://docstring.sourceforge.net/ .. _Docutils: http://docutils.sourceforge.net/ A special thanks must go to Richard Jones for taking up arms (or keyboard) in my stead, and for taking the initiative to start work on `A ReStructuredText Primer`_. In hindsight it was much needed, and it's coming along nicely. Thanks Richard! .. _A ReStructuredText Primer: http://mechanicalcat.net/tech/quickstart.html -- David Goodger goodger@users.sourceforge.net Open-source projects: - Python Docstring Processing System: http://docstring.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net - The Go Tools Project: http://gotools.sourceforge.net From aahz@pythoncraft.com Fri Apr 5 04:38:19 2002 From: aahz@pythoncraft.com (Aahz) Date: Thu, 4 Apr 2002 23:38:19 -0500 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape In-Reply-To: <15533.10206.635597.723733@anthem.wooz.org> References: <3CACE9DF.7E7F199E@activestate.com> <15533.6862.404580.343659@12-248-41-177.client.attbi.com> <15533.10206.635597.723733@anthem.wooz.org> Message-ID: <20020405043819.GA15928@panix.com> On Thu, Apr 04, 2002, Barry A. Warsaw wrote: > > When you're bored, try proxying through junkbuster, which lies about > your browser id. Shut off cookies, Java, and Javascript. Then hunt > around for sites that don't work. Email their webmasters and tell > them that because of your disability you're forced to use a > simplified, non-standard browser, and if they want to comply with the > ADA they'd better make their sites work for folks like you. Don't > tell them that your disability is that you're a bass player that one's for Tim>. Oh, come on, it's easier than that -- just fire up Lynx. (To be honest, I actually like the way we represent such a spectrum, from Tim's witless GUI-centric nature to my hardcore command-line sensibility. ;-) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ I surf faster than you do, monkey boy. (My take on Netscape vs. Lynx) From DavidA@ActiveState.com Fri Apr 5 04:48:52 2002 From: DavidA@ActiveState.com (David Ascher) Date: Thu, 04 Apr 2002 20:48:52 -0800 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape References: Message-ID: <3CAD2CB4.B22100C9@ActiveState.com> Tim Peters wrote: > > [Mark Hammond] > > I don't want to evangelize Mozilla too much, but FWIW I recently > > switched my default browser from IE to Mozilla. > > Dear Lord! When I heard you were starting to use Linux several months ago, > I was afraid something like this might happen. Trust me, Linux had nothing to do with it. If you want to hear an aussie swear, talk to Mark about Linux and GTK =). From barry@zope.com Fri Apr 5 04:53:26 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 4 Apr 2002 23:53:26 -0500 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape References: <3CACE9DF.7E7F199E@activestate.com> <15533.6862.404580.343659@12-248-41-177.client.attbi.com> <15533.10206.635597.723733@anthem.wooz.org> <20020405043819.GA15928@panix.com> Message-ID: <15533.11718.661800.169259@anthem.wooz.org> >>>>> "A" == Aahz writes: A> Oh, come on, it's easier than that -- just fire up Lynx. But then I won't get all the pretty pictures[1]. A> (To be honest, I actually like the way we represent such a A> spectrum, from Tim's witless GUI-centric nature to my hardcore A> command-line sensibility. ;-) Our diversity is our strength. -Barry [1] http://mymomma.will.kick.your.as From David Abrahams" The issue of http://aspn.activestate.com/ASPN/Mail/Message/python-dev/1041123 is back. I had removed Py_TPFLAGS_HAVE_GC from my type, but as Thomas pointed out in http://aspn.activestate.com/ASPN/Mail/Message/python-dev/1039576, the flag is set again by PyType_Ready(). That would be fine if the tp_is_gc field, which is initially zero in my metatype object, were copied automatically from its base (PyTYpe_Type). As it stands, my metatype is "forced" to inherit a claim that it handles GC, but the machinery for determining whether a given instance of the metatype is GC-able is /not/ inherited. Thus, my statically-allocated base instance is treated as though it was allocated by the GC mechanism. This seems unneccessarily dangerous to me, and I propose that we change the semantics of type inheritance to inherit tp_is_gc by default. Further, I think this points out the need for clarification of the subtyping mechanism. There should be one place where I can see which flags and tp_xxx slots are inherited, what they mean, in addition to the answers to other mysteries hidden in typeobject.c et al. I'm willing to work something up, if I can get some guidance as to the most appropriate form. It's not clear to me that there's an appropriate place in the structure of the API docs (it's not even clear to me that the API docs are the right place for this). -Dave P.S. One proviso: I find TeX to be cumbersome at best and hair-pullingly frustrating at worst. If I had to write TeX I probably would just submit code comprehensibility patches and stop there. I'm probably going to start with code patches anyway... +---------------------------------------------------------------+ David Abrahams C++ Booster (http://www.boost.org) O__ == Pythonista (http://www.python.org) c/ /'_ == resume: http://users.rcn.com/abrahams/resume.html (*) \(*) == email: david.abrahams@rcn.com +---------------------------------------------------------------+ From tim.one@comcast.net Fri Apr 5 06:17:13 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 05 Apr 2002 01:17:13 -0500 Subject: [Python-Dev] Basic pymalloc stats Message-ID: FYI, I implemented the optimizations Vladimir and I discussed here. Next, _PyMalloc_DebugDumpStats() is an entry point you can call in a debug build (or when PYMALLOC_DEBUG is enabled in a release build) to get a snapshot of pymalloc's internal structures. Perhaps it should be enabled in a release build too without PYMALLOC_DEBUG -- as is, *because* PYMALLOC_DEBUG is enabled, every allocation is bumped by 16 bytes to make room for PYMALLOC_DEBUG's memory decorations. Here's sample output (recently greatly improved), from near the tail end of a debug-build run of the test suite: Small block threshold = 256, in 32 size classes. pymalloc malloc+realloc called 4414692 times. class num bytes num pools blocks in use avail blocks ----- --------- --------- ------------- ------------ 5 48 773 64932 0 6 56 266 19028 124 7 64 288 18122 22 8 72 124 6914 30 9 80 178 8873 27 10 88 41 1867 19 11 96 28 1170 6 12 104 21 798 21 13 112 16 543 33 14 120 11 359 4 15 128 8 228 20 16 136 5 141 4 17 144 5 114 26 18 152 13 295 43 19 160 6 144 6 20 168 138 3292 20 21 176 5 96 19 22 184 4 76 12 23 192 3 43 20 24 200 3 42 18 25 208 3 40 17 26 216 3 43 11 27 224 2 29 7 28 232 3 32 19 29 240 2 21 11 30 248 2 31 1 31 256 2 21 9 31 arenas * 262144 bytes/arena = 8126464 0 unused pools * 4096 bytes = 0 # bytes in allocated blocks = 7796144 # bytes in available blocks = 69056 # bytes lost to pool headers = 62496 # bytes lost to quantization = 71792 # bytes lost to arena alignment = 126976 Total = 8126464 Running the Unicode tests vastly increases the number of the smallest blocks in use. The hump in the 168-byte class is due to small dicts. Feel lightly encouraged to try calling this in your real programs now, and strongly encouraged after the memory-API rework is complete. Try very hard not to read too much into the test suite . All I take from the above is that memory utilization is excellent; fragmentation is trivial (e.g., in the 56-byte class, 124 available blocks * 56 bytes/block is greater than a 4096-byte pool, so in an ideal world we *could* get away with 265 pools of this size instead of 266); and the wastage due to tossing away "the ends" of arenas to leave pool-aligned pools ("arena alignment") is significant (compared to the other kinds of pure waste in pymalloc -- "quantization" means stuff lost to that the available bytes in a pool often aren't an exact multiple of the pool's block size), but that overall wastage is low. Note that there's no accounting here for what's lost due to returning 8-byte aligned addresses. From martin@v.loewis.de Fri Apr 5 06:23:57 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 05 Apr 2002 08:23:57 +0200 Subject: [Python-Dev] Problem with the memory docs In-Reply-To: References: Message-ID: Tim Peters writes: > Sorry, but this is nuts, and Martin is right that I don't want Python to > offer a "malloc wrapper" that doesn't meet the standard rules for malloc. > Instead I intend to change the docs to say that Py_Malloc(0) acts like > platform malloc(1), and leave it at that. That, OTOH, is not what Guido wants it to be. Guido wants Py_Malloc to return 0 *only* in case of failure; allocating 0 bytes should "normally" succeed. I still cannot see what problems you have with this requirement. Regards, Martin From martin@v.loewis.de Fri Apr 5 06:32:31 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 05 Apr 2002 08:32:31 +0200 Subject: [Python-Dev] New canonical location for PEPs In-Reply-To: References: <15530.31727.557290.92540@grendel.zope.com> <20020402233722.H16962@tummy.com> <20020403004949.I16962@tummy.com> <87ofh0xo93.fsf@tleepslib.sk.tsukuba.ac.jp> <4rishsav.fsf@smtprelay.t-online.de> Message-ID: Adrian Aichner writes: > Hi Martin, where does it say it's disabled or wouldn't work. In CVSROOT/commit-trigger, it says ##### Updating xemacs@shell2 is currently broken. # for trigger_id in xemacs@shell2.sourceforge.net; do # for trigger_id in adrian@sunsite.dk; do # for trigger_id in xemacweb@www.xemacs.org; do # Sourceforge is updated via cvs-commit-trigger-command on www.xemacs.org. # echo "AUTOMATIC WEBSITE UPDATE IS CURRENTLY BROKEN." I couldn't tell what the status of updates to SF is; it says twice that things are broken, but atleast on of them is clearly commented out. > Try a commit-trigger based on ours in combination with an adjusted > authorized_keys entry similar to the one above. I see. So you are forking the cvs update into the background. If that works fine for you, I see whether I can make it to work for us as well. Thanks a lot, Martin From tim.one@comcast.net Fri Apr 5 06:38:19 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 05 Apr 2002 01:38:19 -0500 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape In-Reply-To: <3CAD2CB4.B22100C9@ActiveState.com> Message-ID: [David Ascher, on Mark Hammond's shocking conversion to Netscape] > Trust me, Linux had nothing to do with it. If you want to hear an > aussie swear, talk to Mark about Linux and GTK =). How long did you work with Mark? Getting insanely mad at a system is what sucks him in. All Windows developers are like that. Perl developers too, I hear. Mark's going to write Perl plugins for Netscape under .NET next -- it's inevitable. From tim.one@comcast.net Fri Apr 5 06:45:09 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 05 Apr 2002 01:45:09 -0500 Subject: [Python-Dev] Problem with the memory docs In-Reply-To: Message-ID: >> Sorry, but this is nuts, and Martin is right that I don't want Python >> to offer a "malloc wrapper" that doesn't meet the standard rules >> for malloc. Instead I intend to change the docs to say that Py_Malloc(0) >> acts like platform malloc(1), and leave it at that. [martin@v.loewis.de] > That, OTOH, is not what Guido wants it to be. Guido wants Py_Malloc to > return 0 *only* in case of failure; allocating 0 bytes should > "normally" succeed. I still cannot see what problems you have with > this requirement. I'm not following this at all. Calling malloc(1) does exactly what Guido wants. "malloc(1)" isn't a broken Unix manpage reference, if that's how you're reading it; it means call malloc with an argument of 1. From martin@v.loewis.de Fri Apr 5 06:59:59 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 05 Apr 2002 08:59:59 +0200 Subject: [Python-Dev] GC, flags, and subtyping In-Reply-To: <09ca01c1dc5f$18a16a00$1101a8c0@boostconsulting.com> References: <09ca01c1dc5f$18a16a00$1101a8c0@boostconsulting.com> Message-ID: "David Abrahams" writes: > I'm willing to work something up, if I can get some guidance as to > the most appropriate form. It's not clear to me that there's an > appropriate place in the structure of the API docs (it's not even > clear to me that the API docs are the right place for this). I think the API docs are the right place, probably section 10. Notice that it is very easy to move text from one section to another, so don't worry too much about where this text ends up. More worrying is the fact that new-style classes aren't documented at all. There is nothing wrong, IMO, to start with a piece as obscure as the PyType_Ready function, and I'm not asking that you produce a full newstyle classes specification. However, it would be good if you could allocate a subsection to all aspects of newstyle classes on the C level that you can think of, perhaps filling in the parts that you have been using. Notice that the API documentation currently does not even *list* what the slots in a type object are, or what the possible flags are. Some of it is explained in the embedding and extending tutorial, but that is the wrong place if you want comprehensiveness. So I'd envision the following structure: 10.3: Type Objects Mentions all the slots; subsections elaborate 10.3.1 Mapping Object Structures (what is currently 10.3) ... 10.3.6 GC 10.3.7 Rich Comparisons 10.3.8 Weak references 10.3.9 Descriptors 10.3.10 Object lifecycle 10.4 Inheriting Type Objects As I said, the precise structure can be shifted once the text is there. Hope this can get you started. Regards, Martin From DavidA@ActiveState.com Fri Apr 5 07:09:07 2002 From: DavidA@ActiveState.com (David Ascher) Date: Thu, 04 Apr 2002 23:09:07 -0800 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape References: Message-ID: <3CAD4D93.16FBD52C@ActiveState.com> Tim Peters wrote: > > [David Ascher, on Mark Hammond's shocking conversion to Netscape] > > Trust me, Linux had nothing to do with it. If you want to hear an > > aussie swear, talk to Mark about Linux and GTK =). > > How long did you work with Mark? Two years or so... I don't think Mark has been doing much w/ Linux since he had to. =) > Getting insanely mad at a system is what sucks him in. > All Windows developers are like that. Perl developers too, I > hear. Nah. I work with Perl developers too, and they don't get mad. They get even. This reminds me that I need to get even w/ Mark. He's the one who convinced me to build my own dual-processor machine instead of getting a boring DELL. The machine is now in the shop with a very sick motherboard-which-is-no-longer-being-manufactured-just-after-the-warranty-expired. If Mark builds anything sick, I expect it will be a version of gdb that can debug Mozilla with all the symbols loaded. --da From skip@pobox.com Fri Apr 5 07:37:08 2002 From: skip@pobox.com (Skip Montanaro) Date: Fri, 5 Apr 2002 01:37:08 -0600 Subject: [Python-Dev] autotools online book Message-ID: <15533.21540.762788.930549@12-248-41-177.client.attbi.com> In light of the brief thread yesterday about autoconf, I thought this might be a useful document to make people aware of: http://sources.redhat.com/autobook/autobook/autobook.html It's a largish online book (26 Chapters, 5 Appendices) covering automake, autoconf and libtool, rather more how they work together than simply documenting each tool. I don't know how many people are aware of it. I came across it while struggling with Gtk2 build issues a few months ago and saw mention of it on the tramp mailing list today. Skip From martin@v.loewis.de Fri Apr 5 07:39:31 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 05 Apr 2002 09:39:31 +0200 Subject: [Python-Dev] Problem with the memory docs In-Reply-To: References: Message-ID: Tim Peters writes: > [martin@v.loewis.de] > > That, OTOH, is not what Guido wants it to be. Guido wants Py_Malloc to > > return 0 *only* in case of failure; allocating 0 bytes should > > "normally" succeed. I still cannot see what problems you have with > > this requirement. > > I'm not following this at all. Calling malloc(1) does exactly what Guido > wants. "malloc(1)" isn't a broken Unix manpage reference, if that's how > you're reading it; it means call malloc with an argument of 1. I think I'm now getting to the root of your confusion. Calling malloc(1) is *not* what Guido wants. Instead, he wants that py_malloc(0) returns a non-null pointer "normally". Now, there are some C libraries which normally return a NULL pointer for malloc(0); as you point out, C gives them the freedom to do so. To work-around this under-specification, one can call malloc(1) - standards mandate that this does attempt to allocate something. It may be possible to find different work-arounds, but this is the most straight-forward one. Regards, Martin From fredrik@pythonware.com Fri Apr 5 08:07:23 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Fri, 5 Apr 2002 10:07:23 +0200 Subject: [Python-Dev] PEP 285: Adding a bool type References: Message-ID: <001d01c1dc78$ece213f0$0900a8c0@spiff> Stuart Bishop wrote: > No - I was benchmarking with the patch posted on Sourceforge (before > PEP285 was checked into CVS), patched into a CVS snapshot taken at > the same time. >=20 > Casting to bool using (x and True) or (x or False) : 1.61 > Casting to bool using (bool) : 2.68 > Flow control on explicit True : 1.42 > Flow control on explicit 1 : 0.99 > Flow control on variable bound to True : 1.02 > Flow control on variable bound to 1 : 0.99 time to make keywords out of True, False (and None)? From tim.one@comcast.net Fri Apr 5 10:12:56 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 05 Apr 2002 05:12:56 -0500 Subject: [Python-Dev] Problem with the memory docs In-Reply-To: Message-ID: >> I'm not following this at all. Calling malloc(1) does exactly >> what Guido wants. "malloc(1)" isn't a broken Unix manpage reference, >> if that's how you're reading it; it means call malloc with an argument >> of 1. [Martin] > I think I'm now getting to the root of your confusion. Calling > malloc(1) is *not* what Guido wants. Instead, he wants that > py_malloc(0) returns a non-null pointer "normally". And there's no better way to be 100% clear about what that means than to specify a trivial implementation that satisfies it. It doesn't need a pile of words like this: > Now, there are some C libraries which normally return a NULL pointer > for malloc(0); as you point out, C gives them the freedom to do so. To > work-around this under-specification, one can call malloc(1) - > standards mandate that this does attempt to allocate something. It may > be possible to find different work-arounds, but this is the most > straight-forward one. Writing an essay on malloc() isn't going to make the docs clearer than saying PyMem_Malloc(0) acts like malloc(1). Even the C standard is reduced to saying "the behavior is as if the size were some nonzero value" when a platform chooses not to return NULL every time malloc(0) is called. I'm saying straight out that Python's choice for "some nonzero value" is always 1. It couldn't be clearer or briefer than that: both rigorously correct and intuitively clear with one shot. If there's something you want to see done here, please say what it is, or suggest the exact words you want to see in the docs. From mal@lemburg.com Fri Apr 5 10:49:16 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 05 Apr 2002 12:49:16 +0200 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape References: Message-ID: <3CAD812C.67F70780@lemburg.com> Gerson Kurz wrote: > > I have a version that > allows searching the docs by utilizing google.com, but unfortunately google > license prohibits querying google from a program :( Is that so ? If yes, then we'll have to remove scripts/google.py from the Python distro. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From pedroni@inf.ethz.ch Fri Apr 5 11:33:22 2002 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Fri, 5 Apr 2002 13:33:22 +0200 Subject: [Python-Dev] Re: PEP 287: reStructuredText Standard Docstring Format References: <200204042338.XAA12847@crown.off.ekorp.com> <01c701c1dc35$5a2312f0$02010a0a@suxlap> Message-ID: <005501c1dc95$b17a3aa0$6d94fea9@newmexico> Kent Pitman (KMP) often talks about programming languages as Political Parties (a brilliant intuition). I *honestly* wonder whether to the Python party really corresponds more-or-less just one documentation format party, a silent majority perhaps. And whether Docutils (formerly DPS) will be up to this ambitious design claim in PEP 258 (revision 1.2): Intermediate Data Structure =========================== ... The DTD defines a rich set of elements, suitable for any input syntax or output format or is it already optionless (anyway it is not finished). sometimes-I-wish-I-had-a-real-&-working-Erik-Naggum's-mode- (a-KMP's-mode-would-be-better-and-more-pythonic-but-even-more- ambitious)-ly y'rs. PS: the funny thing is that I do not dislike structured text formats in general, but auto-documenting is reducible to that ? ----- Original Message ----- From: David Goodger Newsgroups: comp.lang.python Cc: Richard Jones ; ; Sent: Friday, April 05, 2002 6:33 AM Subject: [Python-Dev] Re: PEP 287: reStructuredText Standard Docstring Format > Thank you to everyone who has commented on PEP 287, be they in favor, > well-meaning but opposed, or ill-wishing troll ;-) [1]_. I have been > otherwise busy and haven't been able to keep up with replies, but I have > read and kept a record of all the posts so far (and will, for posts yet to > come) for consideration and reference when revising the PEP. I may not be > able to offer timely replies to all the posts; ultimately the revised PEP > will serve as an umbrella reply. > > .. [1] Actually, the posts opposed and even the trolls are often the most > useful, since they point out weaknesses and help strengthen the > arguments. The number of people posting *in favor* of the PEP was > roughly the same as the number opposed, but supporters tend to be > silent while protesters tend to be quite vociferous and relentless. > > Over the weeks to come, I will be revising PEP 287 and related PEPs 256-258, > merging the reStructuredText_ and DPS_ projects into the renamed Docutils_ > project, and producing a "road map" describing the big picture. Further > suggestions for improvement are welcome. > > .. _reStructuredText: http://structuredtext.sourceforge.net/ > .. _DPS: http://docstring.sourceforge.net/ > .. _Docutils: http://docutils.sourceforge.net/ > > A special thanks must go to Richard Jones for taking up arms (or keyboard) > in my stead, and for taking the initiative to start work on `A > ReStructuredText Primer`_. In hindsight it was much needed, and it's coming > along nicely. Thanks Richard! > > .. _A ReStructuredText Primer: http://mechanicalcat.net/tech/quickstart.html > > -- > David Goodger goodger@users.sourceforge.net Open-source projects: > - Python Docstring Processing System: http://docstring.sourceforge.net > - reStructuredText: http://structuredtext.sourceforge.net > - The Go Tools Project: http://gotools.sourceforge.net > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > From mwh@python.net Fri Apr 5 11:55:50 2002 From: mwh@python.net (Michael Hudson) Date: 05 Apr 2002 12:55:50 +0100 Subject: [Python-Dev] Pymalloc and backward compatibility In-Reply-To: "Moore, Paul"'s message of "Thu, 4 Apr 2002 16:54:59 +0100" References: <714DFA46B9BBD0119CD000805FC1F53B01B5B2A6@UKRUX002.rundc.uk.origin-it.com> Message-ID: <2msn6aicl5.fsf@starship.python.net> "Moore, Paul" writes: > Some form of backward compatibility boilerplate which incorporated this > would probably be of use in allowing people to migrate - it might save > everyone writing their own. As I posted a few days ago, I think it would be a good idea if someone (maybe me) wrote a header file that extension authors could bundle with their extensions that would let them program to 2.3 memory APIs and remain compatible all the way back to 1.5.2 (maybe further, if that's easy). Assuming I get this right, does this sound like a good solution? Cheers, M. -- I don't remember any dirty green trousers. -- Ian Jackson, ucam.chat From pedroni@inf.ethz.ch Fri Apr 5 12:56:15 2002 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Fri, 5 Apr 2002 14:56:15 +0200 Subject: [Python-Dev] PEP 287 (politics) Message-ID: <00b701c1dca1$46040b00$6d94fea9@newmexico> Some humanity (which I appreciate) instead of a marketing-machine [Goodger on doc-sig] >This is a classic chicken-and-egg situation. Without tools, we can't >get acceptance, but in this case producing the tools is such a big job >that some form of acceptance up front is needed to keep up enthusiasm >(mine, if nobody else's). One reason for putting this PEP out there >now, is that I'm at a point where I've put a lot of work into this >project, and it's starting to pay off, but I need some reassurance to >keep me going. I want to know that I'm on the right track, that I'm >not wasting my time, and getting this PEP accepted would go a long way >in that direction. > >Even if this PEP is rejected, that will be useful information. If >accepted, then I can forget about supporting multiple markup parsers >in Docutils, at least for now. If rejected, support for multiple >parsers may be its saving grace. now-I-can-better-relate-with-the-guy,-sincerely y'rs, Samuele Pedroni. From Gerson.Kurz@t-online.de Fri Apr 5 14:09:50 2002 From: Gerson.Kurz@t-online.de (Gerson Kurz) Date: Fri, 5 Apr 2002 15:09:50 +0100 Subject: AW: [Python-Dev] Python sidebar for Mozilla/Netscape In-Reply-To: <3CAD812C.67F70780@lemburg.com> Message-ID: > Gerson Kurz wrote: > > > > I have a version that > > allows searching the docs by utilizing google.com, but > unfortunately google > > license prohibits querying google from a program :( > > Is that so ? > > If yes, then we'll have to remove scripts/google.py from > the Python distro. On http://www.google.com/webmasters/dos.html it says in the "DONT" section: "Send automated queries to Google in an attempt to monitor your site's ranking." Rereading that I can interpret that as a) no automatic query at all b) no automatic query if only for the purpose of site ranking c) its just a "please don't" thing, not something the license prohibits I'll mail them and see what they answer. Bye, Gerson From ark@research.att.com Fri Apr 5 14:08:18 2002 From: ark@research.att.com (Andrew Koenig) Date: 05 Apr 2002 09:08:18 -0500 Subject: [Python-Dev] PEP 285: Adding a bool type In-Reply-To: References: Message-ID: Stuart> On Friday, April 5, 2002, at 01:06 AM, Guido van Rossum wrote: >> On Thursday, April 4, 2002, at 02:22 PM, Stuart Bishop wrote: >>> >>>> On a side note, writing 'i or False and True' performs much better Stuart> I mean (i and True) or (i or False) This example might evaluate i twice, which means it's not really a fair comparison. -- Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark From aahz@pythoncraft.com Fri Apr 5 14:13:58 2002 From: aahz@pythoncraft.com (Aahz) Date: Fri, 5 Apr 2002 09:13:58 -0500 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape In-Reply-To: References: <3CAD812C.67F70780@lemburg.com> Message-ID: <20020405141358.GA23056@panix.com> On Fri, Apr 05, 2002, Gerson Kurz wrote: >> Gerson Kurz wrote: >>> >>> I have a version that allows searching the docs by utilizing >>> google.com, but unfortunately google license prohibits querying >>> google from a program :( >> >> Is that so ? >> >> If yes, then we'll have to remove scripts/google.py from >> the Python distro. > > On http://www.google.com/webmasters/dos.html it says in the "DONT" section: > "Send automated queries to Google in an attempt to monitor your site's > ranking." That just means that you shouldn't use cron jobs and the like; it's perfectly fine to have a human-initiated automated interaction. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From thomas.heller@ion-tof.com Fri Apr 5 15:03:15 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri, 5 Apr 2002 17:03:15 +0200 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) References: <3CAC2F2B.1950AD8F@lemburg.com><018e01c1dbee$51b1c9f0$e000a8c0@thomasnotebook> <15532.48440.566653.321044@magrathea.basistech.com> Message-ID: <025701c1dcb3$037af110$e000a8c0@thomasnotebook> From: "Tom Emerson" > Thomas Heller writes: > > I'm interested in feedback: Is it usefull at all? Should > > something like this go into the Python distribution? How can it be > > improved? What should be the user interface - currently it is command > > line only and usable from the interactive interpreter programmatically. > > Cool: I had to modify it slightly to run correctly on my system > (FreeBSD 4.4 with Lynx) (change below) but otherwise it's great. I've implemented your change, and I have already extended it to also work as a CGI script. The preliminary version is online at http://starship.python.net/crew/theller/cgi-bin/pyhelp.cgi Anyway, at least on one question I would really like to get more answers : when writing python code or C extensions, how do people quickly look up definitions of functions, let's say PyObject_GC_New() or the meaning of METH_O ? Thanks, Thomas From aahz@pythoncraft.com Fri Apr 5 15:15:19 2002 From: aahz@pythoncraft.com (Aahz) Date: Fri, 5 Apr 2002 10:15:19 -0500 Subject: [Python-Dev] Pymalloc and backward compatibility In-Reply-To: <714DFA46B9BBD0119CD000805FC1F53B01B5B2A4@UKRUX002.rundc.uk.origin-it.com> References: <714DFA46B9BBD0119CD000805FC1F53B01B5B2A4@UKRUX002.rundc.uk.origin-it.com> Message-ID: <20020405151519.GA4682@panix.com> On Thu, Apr 04, 2002, Moore, Paul wrote: > > OK, so maybe it's time to give up on Python 1.4 compatibility. But > ideally, I'd prefer to go for an approach which leaves the code used > under Python 2.2 and earlier completely unchanged, and only adds the > minimum necessary for newer versions. > > Longer term, it might be worth considering upgrading the code to use > some of the newer features in the Python API, but in all honesty > that's not a great priority for me (and unlikely to be even remotely > on the cards until I can realistically desupport everything before > 2.0...) Why can't you realistically desupport pre-2.0? Believe me, I understand all the arguments in favor of keeping support for old versions, but someone who's getting a *new* version of vim has IMO demonstrably indicated that zie wishes to get access to new software. Even if zie needs to support old versions of Python, why shouldn't zie use a current version as the macro language in vim? What I'd suggest doing for your case is targetting Python 2.2 without pyMalloc, and just making sure you follow the 2.3 rules. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From Gerson.Kurz@t-online.de Fri Apr 5 16:38:40 2002 From: Gerson.Kurz@t-online.de (Gerson Kurz) Date: Fri, 5 Apr 2002 17:38:40 +0100 Subject: AW: [Python-Dev] Python sidebar for Mozilla/Netscape In-Reply-To: <20020405141358.GA23056@panix.com> Message-ID: Aahz wrote: > That just means that you shouldn't use cron jobs and the like; it's > perfectly fine to have a human-initiated automated interaction. ok, then try http://www.p-nand-q.com/cgi-bin/zgidocs.py ;) it has the google search included. From guido@python.org Fri Apr 5 16:06:04 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 11:06:04 -0500 Subject: [Python-Dev] GC, flags, and subtyping In-Reply-To: Your message of "Fri, 05 Apr 2002 00:02:17 EST." <09ca01c1dc5f$18a16a00$1101a8c0@boostconsulting.com> References: <09ca01c1dc5f$18a16a00$1101a8c0@boostconsulting.com> Message-ID: <200204051606.g35G65B13089@pcp742651pcs.reston01.va.comcast.net> > The issue of > http://aspn.activestate.com/ASPN/Mail/Message/python-dev/1041123 is > back. (BTW, the ASPN archives of python-dev are much prettier-looking than the pipermail ones, and they are searchable. Maybe Barry or Jeremy could update the list info to also point to the ASPN archives?) > I had removed Py_TPFLAGS_HAVE_GC from my type, but as Thomas pointed out > in http://aspn.activestate.com/ASPN/Mail/Message/python-dev/1039576, > the flag is set again by PyType_Ready(). That would be fine if the > tp_is_gc field, which is initially zero in my metatype object, were > copied automatically from its base (PyTYpe_Type). As it stands, my > metatype is "forced" to inherit a claim that it handles GC, but the > machinery for determining whether a given instance of the metatype > is GC-able is /not/ inherited. Thus, my statically-allocated base > instance is treated as though it was allocated by the GC mechanism. Note that if a subtype provides its own tp_traverse and/or tp_clear, it does *not* inherit the HAVE_GC flag. (I'm not sure if this helps.) > This seems unneccessarily dangerous to me, and I propose that we > change the semantics of type inheritance to inherit tp_is_gc by > default. Agreed. I think this was purely an oversight. Looking at the end of inherit_slots(), I notice that tp_new isn't inherited either -- I think that needs to be added to the list of inherited slots too. > Further, I think this points out the need for clarification of the > subtyping mechanism. There should be one place where I can see which > flags and tp_xxx slots are inherited, what they mean, in addition to > the answers to other mysteries hidden in typeobject.c et al. I'm > willing to work something up, if I can get some guidance as to the > most appropriate form. It's not clear to me that there's an > appropriate place in the structure of the API docs (it's not even > clear to me that the API docs are the right place for this). The API docs *are* the right place. If you hate LaTeX, you can write the docs as plain text and send them to the appropriate channels -- they will magically be turned into LaTeX. (Maybe you can give reST a try? I'm sure that at some point we'll need an automatic translator to py-doc-latex, so we might as well get started on that now.) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Apr 5 16:13:25 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 11:13:25 -0500 Subject: [Python-Dev] Problem with the memory docs In-Reply-To: Your message of "05 Apr 2002 08:23:57 +0200." References: Message-ID: <200204051613.g35GDPR13375@pcp742651pcs.reston01.va.comcast.net> > > Sorry, but this is nuts, and Martin is right that I don't want Python to > > offer a "malloc wrapper" that doesn't meet the standard rules for malloc. > > Instead I intend to change the docs to say that Py_Malloc(0) acts like > > platform malloc(1), and leave it at that. > > That, OTOH, is not what Guido wants it to be. Guido wants Py_Malloc to > return 0 *only* in case of failure; allocating 0 bytes should > "normally" succeed. I still cannot see what problems you have with > this requirement. How is this different from what Tim says? Maybe you read what Tim wrote as malloc(0)? But he wrote malloc(1). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Apr 5 16:14:29 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 11:14:29 -0500 Subject: [Python-Dev] GC, flags, and subtyping In-Reply-To: Your message of "05 Apr 2002 08:59:59 +0200." References: <09ca01c1dc5f$18a16a00$1101a8c0@boostconsulting.com> Message-ID: <200204051614.g35GETF13388@pcp742651pcs.reston01.va.comcast.net> > More worrying is the fact that new-style classes aren't documented at > all. Fixing this is actually one of PythonLabs' priorities at the moment. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Apr 5 16:18:41 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 11:18:41 -0500 Subject: [Python-Dev] Problem with the memory docs In-Reply-To: Your message of "05 Apr 2002 09:39:31 +0200." References: Message-ID: <200204051618.g35GIgd13439@pcp742651pcs.reston01.va.comcast.net> > Tim Peters writes: > > > [martin@v.loewis.de] > > > That, OTOH, is not what Guido wants it to be. Guido wants Py_Malloc to > > > return 0 *only* in case of failure; allocating 0 bytes should > > > "normally" succeed. I still cannot see what problems you have with > > > this requirement. > > > > I'm not following this at all. Calling malloc(1) does exactly what Guido > > wants. "malloc(1)" isn't a broken Unix manpage reference, if that's how > > you're reading it; it means call malloc with an argument of 1. > > I think I'm now getting to the root of your confusion. Calling > malloc(1) is *not* what Guido wants. Instead, he wants that > py_malloc(0) returns a non-null pointer "normally". > > Now, there are some C libraries which normally return a NULL pointer > for malloc(0); as you point out, C gives them the freedom to do so. To > work-around this under-specification, one can call malloc(1) - > standards mandate that this does attempt to allocate something. It may > be possible to find different work-arounds, but this is the most > straight-forward one. I'm still confused. What is the difference between calling malloc(1) and returning a non-null pointer "normally" ? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Apr 5 16:21:44 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 11:21:44 -0500 Subject: [Python-Dev] PEP 285: Adding a bool type In-Reply-To: Your message of "Fri, 05 Apr 2002 10:07:23 +0200." <001d01c1dc78$ece213f0$0900a8c0@spiff> References: <001d01c1dc78$ece213f0$0900a8c0@spiff> Message-ID: <200204051621.g35GLiW13454@pcp742651pcs.reston01.va.comcast.net> > > No - I was benchmarking with the patch posted on Sourceforge (before > > PEP285 was checked into CVS), patched into a CVS snapshot taken at > > the same time. > > > > Casting to bool using (x and True) or (x or False) : 1.61 > > Casting to bool using (bool) : 2.68 > > Flow control on explicit True : 1.42 > > Flow control on explicit 1 : 0.99 > > Flow control on variable bound to True : 1.02 > > Flow control on variable bound to 1 : 0.99 > > time to make keywords out of True, False (and None)? None should be a keyword, yes. Patch anyone? Making True and False keywords would cause unnecessary breakage of legacy code that defines its own bools and names them True or False. We might be able to do this after a few more releases, after we've added warnings for assignment to them. I kind of doubt that calls to bool() or code saying "if True" (or "return True" or "x = True") is in the critical path of anything. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Apr 5 16:25:44 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 11:25:44 -0500 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape In-Reply-To: Your message of "Fri, 05 Apr 2002 12:49:16 +0200." <3CAD812C.67F70780@lemburg.com> References: <3CAD812C.67F70780@lemburg.com> Message-ID: <200204051625.g35GPib13488@pcp742651pcs.reston01.va.comcast.net> > > I have a version that allows searching the docs by utilizing > > google.com, but unfortunately google license prohibits querying > > google from a program :( > > Is that so ? I think it is. Read http://www.google.com/terms_of_service.html > If yes, then we'll have to remove scripts/google.py from > the Python distro. I don't think so -- the google.py code doesn't read the Google output, it tells your browser to read it. Big difference. (urlopen(), on the other hand, would be a no-no.) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Apr 5 16:28:01 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 11:28:01 -0500 Subject: [Python-Dev] Pymalloc and backward compatibility In-Reply-To: Your message of "05 Apr 2002 12:55:50 +0100." <2msn6aicl5.fsf@starship.python.net> References: <714DFA46B9BBD0119CD000805FC1F53B01B5B2A6@UKRUX002.rundc.uk.origin-it.com> <2msn6aicl5.fsf@starship.python.net> Message-ID: <200204051628.g35GS1H13516@pcp742651pcs.reston01.va.comcast.net> > As I posted a few days ago, I think it would be a good idea if someone > (maybe me) wrote a header file that extension authors could bundle > with their extensions that would let them program to 2.3 memory APIs > and remain compatible all the way back to 1.5.2 (maybe further, if > that's easy). > > Assuming I get this right, does this sound like a good solution? Yes! --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Apr 5 16:31:01 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 11:31:01 -0500 Subject: [Python-Dev] PEP 285: Adding a bool type In-Reply-To: Your message of "05 Apr 2002 09:08:18 EST." References: Message-ID: <200204051631.g35GV1k13573@pcp742651pcs.reston01.va.comcast.net> > Stuart> I mean (i and True) or (i or False) > > This example might evaluate i twice, which means it's not really a > fair comparison. But the correct form (i and True or False) still performs faster than bool(i). I'm not sure I care though. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Fri Apr 5 16:33:58 2002 From: skip@pobox.com (Skip Montanaro) Date: Fri, 5 Apr 2002 10:33:58 -0600 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape In-Reply-To: <200204051625.g35GPib13488@pcp742651pcs.reston01.va.comcast.net> References: <3CAD812C.67F70780@lemburg.com> <200204051625.g35GPib13488@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15533.53750.930189.744291@beluga.mojam.com> Guido> I don't think so -- the google.py code doesn't read the Google Guido> output, it tells your browser to read it. Big difference. Guido> (urlopen(), on the other hand, would be a no-no.) Correct. I think they want to avoid automated systems that tweak a webpage, see how placement is affected, tweak it again, etc. Also, if you notice their links, they redirect through Google itself: http://www.google.com/url?sa=U&start=1&q=http://www.python.org/&e=921 (I don't think they used to do this, but I'm not certain.) I'm sure they use click frequency as a component of their relevance algorithm. A program running through those links would adversely affect their scoring algorithm. Skip From aahz@pythoncraft.com Fri Apr 5 16:32:36 2002 From: aahz@pythoncraft.com (Aahz) Date: Fri, 5 Apr 2002 11:32:36 -0500 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape In-Reply-To: <200204051625.g35GPib13488@pcp742651pcs.reston01.va.comcast.net> References: <3CAD812C.67F70780@lemburg.com> <200204051625.g35GPib13488@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020405163236.GA24796@panix.com> Disclaimer: IANAL On Fri, Apr 05, 2002, Guido van Rossum wrote: > >>> I have a version that allows searching the docs by utilizing >>> google.com, but unfortunately google license prohibits querying >>> google from a program :( >> >> Is that so ? > > I think it is. Read http://www.google.com/terms_of_service.html Now that I've read the entire thing, I have to agree that it's stricter than the little snippet posted earlier... >> If yes, then we'll have to remove scripts/google.py from >> the Python distro. > > I don't think so -- the google.py code doesn't read the Google output, > it tells your browser to read it. Big difference. (urlopen(), on the > other hand, would be a no-no.) ...but I disagree with your assertion about urlopen() -- as long as the output gets rendered directly by a browser-like interface for on-line interactive human use, I believe there's no contravention of the ToS. (If your assertion were correct, one wouldn't be able to write a browser in Python. ;-) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From mal@lemburg.com Fri Apr 5 16:44:07 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 05 Apr 2002 18:44:07 +0200 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) References: <3CAC2F2B.1950AD8F@lemburg.com><018e01c1dbee$51b1c9f0$e000a8c0@thomasnotebook> <15532.48440.566653.321044@magrathea.basistech.com> <025701c1dcb3$037af110$e000a8c0@thomasnotebook> Message-ID: <3CADD457.4D821311@lemburg.com> Thomas Heller wrote: > > From: "Tom Emerson" > > Thomas Heller writes: > > > I'm interested in feedback: Is it usefull at all? Should > > > something like this go into the Python distribution? How can it be > > > improved? What should be the user interface - currently it is command > > > line only and usable from the interactive interpreter programmatically. > > > > Cool: I had to modify it slightly to run correctly on my system > > (FreeBSD 4.4 with Lynx) (change below) but otherwise it's great. > > I've implemented your change, and I have already extended it to > also work as a CGI script. The preliminary version is online at > http://starship.python.net/crew/theller/cgi-bin/pyhelp.cgi > > Anyway, at least on one question I would really like to get more > answers : when writing python code or C extensions, how do people > quickly look up definitions of functions, let's say PyObject_GC_New() > or the meaning of METH_O ? grep in the Include/ dir, possibly followed by a less session :-) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From mal@lemburg.com Fri Apr 5 16:52:34 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 05 Apr 2002 18:52:34 +0200 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape References: <3CAD812C.67F70780@lemburg.com> <200204051625.g35GPib13488@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CADD652.535193C5@lemburg.com> Guido van Rossum wrote: > > > > I have a version that allows searching the docs by utilizing > > > google.com, but unfortunately google license prohibits querying > > > google from a program :( > > > > Is that so ? > > I think it is. Read http://www.google.com/terms_of_service.html > > > If yes, then we'll have to remove scripts/google.py from > > the Python distro. > > I don't think so -- the google.py code doesn't read the Google output, > it tells your browser to read it. Big difference. (urlopen(), on the > other hand, would be a no-no.) I see. False alarm, then. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From guido@python.org Fri Apr 5 17:21:07 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 12:21:07 -0500 Subject: [Python-Dev] Pymalloc and backward compatibility In-Reply-To: Your message of "Fri, 05 Apr 2002 10:15:19 EST." <20020405151519.GA4682@panix.com> References: <714DFA46B9BBD0119CD000805FC1F53B01B5B2A4@UKRUX002.rundc.uk.origin-it.com> <20020405151519.GA4682@panix.com> Message-ID: <200204051721.g35HL7P14646@pcp742651pcs.reston01.va.comcast.net> > Why can't you realistically desupport pre-2.0? Because 1.5.2 is still most commonly found as the default Python, like it or not. I don't care for 1.4, but 1.5.2 support is a MUST. --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@pythoncraft.com Fri Apr 5 17:25:48 2002 From: aahz@pythoncraft.com (Aahz) Date: Fri, 5 Apr 2002 12:25:48 -0500 Subject: [Python-Dev] Pymalloc and backward compatibility In-Reply-To: <200204051721.g35HL7P14646@pcp742651pcs.reston01.va.comcast.net> References: <714DFA46B9BBD0119CD000805FC1F53B01B5B2A4@UKRUX002.rundc.uk.origin-it.com> <20020405151519.GA4682@panix.com> <200204051721.g35HL7P14646@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020405172548.GA8114@panix.com> On Fri, Apr 05, 2002, Guido van Rossum wrote: > Aahz: >> >> Why can't you realistically desupport pre-2.0? > > Because 1.5.2 is still most commonly found as the default Python, like > it or not. I don't care for 1.4, but 1.5.2 support is a MUST. I must be missing something here. We're not talking about an end-user product, we're talking about a developer tool. What's wrong with requiring a Python download for someone who at this point wants to *upgrade* vim from the system default? The default Python for Mandrake 8.1 (*not* the most current version), for example, is Python 2.1.1. I don't think Python 1.5.2 is the default Python for anything other than Red Hat at the moment, and even if I'm wrong about that now, it certainly will be true by the time any new version of vim becomes an installed default. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From guido@python.org Fri Apr 5 17:32:38 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 12:32:38 -0500 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape In-Reply-To: Your message of "Fri, 05 Apr 2002 10:33:58 CST." <15533.53750.930189.744291@beluga.mojam.com> References: <3CAD812C.67F70780@lemburg.com> <200204051625.g35GPib13488@pcp742651pcs.reston01.va.comcast.net> <15533.53750.930189.744291@beluga.mojam.com> Message-ID: <200204051732.g35HWcn14816@pcp742651pcs.reston01.va.comcast.net> > Correct. I think they want to avoid automated systems that tweak a > webpage, see how placement is affected, tweak it again, etc. That would take forever (you'd have to wait for a second visit from their crawler). I think they want to avoid added-value websites that don't mention the Google brand -- it would be like redistributing Python but claiming you wrote it yourself. > Also, if you notice > their links, they redirect through Google itself: > > http://www.google.com/url?sa=U&start=1&q=http://www.python.org/&e=921 > > (I don't think they used to do this, but I'm not certain.) Me too. > I'm sure they use click frequency as a component of their relevance > algorithm. A program running through those links would adversely > affect their scoring algorithm. Indeed. And I don't want to affect their scoring algorithm -- it's their most valuable feature! --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Apr 5 17:34:39 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 12:34:39 -0500 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape In-Reply-To: Your message of "Fri, 05 Apr 2002 11:32:36 EST." <20020405163236.GA24796@panix.com> References: <3CAD812C.67F70780@lemburg.com> <200204051625.g35GPib13488@pcp742651pcs.reston01.va.comcast.net> <20020405163236.GA24796@panix.com> Message-ID: <200204051734.g35HYdK14836@pcp742651pcs.reston01.va.comcast.net> > > I don't think so -- the google.py code doesn't read the Google output, > > it tells your browser to read it. Big difference. (urlopen(), on the > > other hand, would be a no-no.) > > ...but I disagree with your assertion about urlopen() -- as long as the > output gets rendered directly by a browser-like interface for on-line > interactive human use, I believe there's no contravention of the ToS. > (If your assertion were correct, one wouldn't be able to write a browser > in Python. ;-) Yeah. I should probably have said "a program that uses urlopen and then parses the page it gets back for a different purpose than rendering it to a human". --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Fri Apr 5 17:32:24 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 05 Apr 2002 19:32:24 +0200 Subject: [Python-Dev] Updating to autoconf 2.5x References: <200204041547.g34Flflv007093@paros.informatik.hu-berlin.de> <15532.31123.535297.863287@12-248-41-177.client.attbi.com> <3CAC8141.83A323C4@lemburg.com> <200204041727.g34HRIR05506@odiug.zope.com> Message-ID: <3CADDFA8.88AB3293@lemburg.com> Guido van Rossum wrote: > > > Side note: I recently found that the socket module is using > > bleeding edge glibc APIs as well, in fact it was the only > > module that needed the most recent glibc version installed > > on my machine (2.1.2). > > Which API? Rechecking the .so file with ldd and nm showed that the problem was due to the linker using a more current version of gethostbyname_r it found in GLIBC 2.1.2 rather than the one in GLIBC 2.1. > > All other modules were happy with plain glibc 2.1. I found > > out about this when installing an application on RedHat which > > didn't have glibc 2.1.2 installed. > > > > This may be a marginal problem, but do we really need to > > live on the bleeding edge of C libraries ? And if so, is > > there a way to configure Python to only use, say, glibc 2.1 > > APIs (to enhance binary compatibility) ? > > If we're using bleeding edge APIs, they should be properly tested for > in configure.in and the code should use #ifdef. Agreed. In this case, I think, it was a false alarm though, since AFAIK, we can't tell the linker to only use one GLIBC version (_socket.so links to APIs from three different versions on my machine: GLIBC 2.0, 2.1 and 2.1.2). BTW, the man-page for ld mentions an optimization option -On -- does anyone know what the affect of this option is and whether its GNU specific ? -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From mal@lemburg.com Fri Apr 5 17:41:30 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 05 Apr 2002 19:41:30 +0200 Subject: [Python-Dev] Pymalloc and backward compatibility References: <714DFA46B9BBD0119CD000805FC1F53B01B5B2A4@UKRUX002.rundc.uk.origin-it.com> <20020405151519.GA4682@panix.com> <200204051721.g35HL7P14646@pcp742651pcs.reston01.va.comcast.net> <20020405172548.GA8114@panix.com> Message-ID: <3CADE1CA.856D38B8@lemburg.com> Aahz wrote: > > On Fri, Apr 05, 2002, Guido van Rossum wrote: > > Aahz: > >> > >> Why can't you realistically desupport pre-2.0? > > > > Because 1.5.2 is still most commonly found as the default Python, like > > it or not. I don't care for 1.4, but 1.5.2 support is a MUST. > > I must be missing something here. We're not talking > about an end-user product, we're talking about a developer tool. What's > wrong with requiring a Python download for someone who at this point > wants to *upgrade* vim from the system default? > > The default Python for Mandrake 8.1 (*not* the most current version), for > example, is Python 2.1.1. I don't think Python 1.5.2 is the default > Python for anything other than Red Hat at the moment, and even if I'm > wrong about that now, it certainly will be true by the time any new > version of vim becomes an installed default. Python 1.5.2 is still in active use. Some of the extension writers simply can't drop 1.5.2 because of an active user base still working with it (either because they are using commercial products built on top of 1.5.2 or because they like the speed... Python 2.x is slower than 1.5.2, or because their books still reference the old version). Python's release schedule has changed dramatically over the last few years -- I would expect that the Python 2.0 user base will also last a few more years. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From guido@python.org Fri Apr 5 17:46:39 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 12:46:39 -0500 Subject: [Python-Dev] Pymalloc and backward compatibility In-Reply-To: Your message of "Fri, 05 Apr 2002 12:25:48 EST." <20020405172548.GA8114@panix.com> References: <714DFA46B9BBD0119CD000805FC1F53B01B5B2A4@UKRUX002.rundc.uk.origin-it.com> <20020405151519.GA4682@panix.com> <200204051721.g35HL7P14646@pcp742651pcs.reston01.va.comcast.net> <20020405172548.GA8114@panix.com> Message-ID: <200204051746.g35HkdG14960@pcp742651pcs.reston01.va.comcast.net> > >> Why can't you realistically desupport pre-2.0? > > > > Because 1.5.2 is still most commonly found as the default Python, like > > it or not. I don't care for 1.4, but 1.5.2 support is a MUST. > > I must be missing something here. We're not talking > about an end-user product, we're talking about a developer tool. > What's wrong with requiring a Python download for someone who at > this point wants to *upgrade* vim from the system default? I wasn't talking about vim, sorry. But we should support Python developers who want to write extensions that work with all versions from Python 1.5.2 onwards. > The default Python for Mandrake 8.1 (*not* the most current > version), for example, is Python 2.1.1. I don't think Python 1.5.2 > is the default Python for anything other than Red Hat at the moment, > and even if I'm wrong about that now, it certainly will be true by > the time any new version of vim becomes an installed default. Vendors can release new versions as much as they want, but most users will only upgrade when forced. Python 1.5.2 binaries are still *very* common. I still get mail from people who say "why do you change Python so much? 1.5.2 works just fine for me." --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@pythoncraft.com Fri Apr 5 17:44:55 2002 From: aahz@pythoncraft.com (Aahz) Date: Fri, 5 Apr 2002 12:44:55 -0500 Subject: [Python-Dev] Pymalloc and backward compatibility In-Reply-To: <3CADE1CA.856D38B8@lemburg.com> References: <714DFA46B9BBD0119CD000805FC1F53B01B5B2A4@UKRUX002.rundc.uk.origin-it.com> <20020405151519.GA4682@panix.com> <200204051721.g35HL7P14646@pcp742651pcs.reston01.va.comcast.net> <20020405172548.GA8114@panix.com> <3CADE1CA.856D38B8@lemburg.com> Message-ID: <20020405174455.GA14217@panix.com> On Fri, Apr 05, 2002, M.-A. Lemburg wrote: > Aahz wrote: >> On Fri, Apr 05, 2002, Guido van Rossum wrote: >>> Aahz: >>>> >>>> Why can't you realistically desupport pre-2.0? >>> >>> Because 1.5.2 is still most commonly found as the default Python, like >>> it or not. I don't care for 1.4, but 1.5.2 support is a MUST. >> >> I must be missing something here. We're not talking >> about an end-user product, we're talking about a developer tool. What's >> wrong with requiring a Python download for someone who at this point >> wants to *upgrade* vim from the system default? >> >> The default Python for Mandrake 8.1 (*not* the most current version), for >> example, is Python 2.1.1. I don't think Python 1.5.2 is the default >> Python for anything other than Red Hat at the moment, and even if I'm >> wrong about that now, it certainly will be true by the time any new >> version of vim becomes an installed default. > > Python 1.5.2 is still in active use. Some of the extension writers > simply can't drop 1.5.2 because of an active user base still working > with it (either because they are using commercial products built on > top of 1.5.2 or because they like the speed... Python 2.x is slower > than 1.5.2, or because their books still reference the old version). > > Python's release schedule has changed dramatically over the last few > years -- I would expect that the Python 2.0 user base will also last a > few more years. Yes, yes, yes, yes, yes, yes, yes -- but why is this a problem for vim? I am not talking about some abstract general case; it looks to me that in this specific case backward compatibility isn't an issue, and I still have not seen any explanation for why I'm wrong. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From aahz@pythoncraft.com Fri Apr 5 17:48:33 2002 From: aahz@pythoncraft.com (Aahz) Date: Fri, 5 Apr 2002 12:48:33 -0500 Subject: [Python-Dev] Pymalloc and backward compatibility In-Reply-To: <200204051746.g35HkdG14960@pcp742651pcs.reston01.va.comcast.net> References: <714DFA46B9BBD0119CD000805FC1F53B01B5B2A4@UKRUX002.rundc.uk.origin-it.com> <20020405151519.GA4682@panix.com> <200204051721.g35HL7P14646@pcp742651pcs.reston01.va.comcast.net> <20020405172548.GA8114@panix.com> <200204051746.g35HkdG14960@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020405174833.GB14217@panix.com> On Fri, Apr 05, 2002, Guido van Rossum wrote: > >>>> Why can't you realistically desupport pre-2.0? >>> >>> Because 1.5.2 is still most commonly found as the default Python, like >>> it or not. I don't care for 1.4, but 1.5.2 support is a MUST. >> >> I must be missing something here. We're not talking >> about an end-user product, we're talking about a developer tool. >> What's wrong with requiring a Python download for someone who at >> this point wants to *upgrade* vim from the system default? > > I wasn't talking about vim, sorry. But we should support Python > developers who want to write extensions that work with all versions > from Python 1.5.2 onwards. With that I can whole-heartedly agree (after all, I'm writing the BCD module to be backward compatible, and I had to muck with some of Tim's code to do it ;-). >> The default Python for Mandrake 8.1 (*not* the most current >> version), for example, is Python 2.1.1. I don't think Python 1.5.2 >> is the default Python for anything other than Red Hat at the moment, >> and even if I'm wrong about that now, it certainly will be true by >> the time any new version of vim becomes an installed default. > > Vendors can release new versions as much as they want, but most users > will only upgrade when forced. Python 1.5.2 binaries are still *very* > common. I still get mail from people who say "why do you change > Python so much? 1.5.2 works just fine for me." I just think the criteria should be a bit different for developer tools than end-user products, particularly in the context of an upgraded developer tool being dependent on a new version of Python. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From tim.one@comcast.net Fri Apr 5 18:00:26 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 05 Apr 2002 13:00:26 -0500 Subject: [Python-Dev] Pymalloc and backward compatibility In-Reply-To: <20020405174455.GA14217@panix.com> Message-ID: [Aahz] > ... > I am not talking about some abstract general case; it looks to me that in > this specific case backward compatibility isn't an issue, and I still > have not seen any explanation for why I'm wrong. Well, give us a reason to believe you're right . Specific cases can't be decided on "general principles" -- the only way to know what vim's needs actually are is to study its source code. Have you done that? I haven't. From skip@pobox.com Fri Apr 5 18:17:02 2002 From: skip@pobox.com (Skip Montanaro) Date: Fri, 5 Apr 2002 12:17:02 -0600 Subject: [Python-Dev] What google will let you do In-Reply-To: <200204051732.g35HWcn14816@pcp742651pcs.reston01.va.comcast.net> References: <3CAD812C.67F70780@lemburg.com> <200204051625.g35GPib13488@pcp742651pcs.reston01.va.comcast.net> <15533.53750.930189.744291@beluga.mojam.com> <200204051732.g35HWcn14816@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15533.59934.996300.701115@beluga.mojam.com> >> Correct. I think they want to avoid automated systems that tweak a >> webpage, see how placement is affected, tweak it again, etc. Guido> That would take forever (you'd have to wait for a second visit Guido> from their crawler). Not necessarily, and of course, it depends on your definition of "forever". I suspect they probably have their crawler coded to visit files more often that change frequently (like freshmeat.net or slashdot.org). It's not practical to input change frequency info by humans, so the crawler probably realizes that a page changed since the last time it was crawled and decides on its own to shorten up the interval before the next check. (They would probably have it lengthen the interval if the file didn't change as well.) They can thus adjust their visit frequency to keep track of pages that change rapidly, but not waste time crawling static pages. The click frequency may also figure into this. A page that is visited more often probably warrants closer attention by the googlebot. I suspect a person wanting to manipulate his site's google ranking could twiddle his page and get feedback on the results within a few days. Over the course of a month or two, he could probably make substantial progress on raising his Google quotient. Guido> I think they want to avoid added-value websites that don't Guido> mention the Google brand -- it would be like redistributing Guido> Python but claiming you wrote it yourself. That too. They explicitly deny meta-search access and even say "don't even ask - your request will be denied", or words to that effect. So, we've concluded nothing, but have given our frontal cortex its speculative workout for the day... ;-) Skip From martin@v.loewis.de Fri Apr 5 18:21:19 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 05 Apr 2002 20:21:19 +0200 Subject: [Python-Dev] Problem with the memory docs In-Reply-To: <200204051618.g35GIgd13439@pcp742651pcs.reston01.va.comcast.net> References: <200204051618.g35GIgd13439@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > I'm still confused. What is the difference between calling malloc(1) > and returning a non-null pointer "normally" ? There is probably no practical difference. If you do malloc(0) and get a non-zero pointer, you cannot access any memory behind this pointer; if you invoke malloc(1), the C library must make sure that there is atleast a single byte to access. This may take larger efforts - but probably doesn't, in practice (unless your malloc implementation tries to clear the memory it allocates with a fence pattern). Regards, Martin From thomas.heller@ion-tof.com Fri Apr 5 18:21:44 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri, 5 Apr 2002 20:21:44 +0200 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) References: <3CAC2F2B.1950AD8F@lemburg.com><018e01c1dbee$51b1c9f0$e000a8c0@thomasnotebook> <15532.48440.566653.321044@magrathea.basistech.com> <025701c1dcb3$037af110$e000a8c0@thomasnotebook> <3CADD457.4D821311@lemburg.com> Message-ID: <03e401c1dcce$bdb7fd50$e000a8c0@thomasnotebook> > > Anyway, at least on one question I would really like to get more > > answers : when writing python code or C extensions, how do people > > quickly look up definitions of functions, let's say PyObject_GC_New() > > or the meaning of METH_O ? > > grep in the Include/ dir, > possibly followed by a less session :-) > Hm, in the good old days this was the only way ;-) because the chances where at most 50% that this stuff was undocumented. But the docs have improved in the meantime, now you could also use *them*. Thomas From martin@v.loewis.de Fri Apr 5 18:18:35 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 05 Apr 2002 20:18:35 +0200 Subject: [Python-Dev] Problem with the memory docs In-Reply-To: References: Message-ID: Tim Peters writes: > [Martin] > > I think I'm now getting to the root of your confusion. Calling > > malloc(1) is *not* what Guido wants. Instead, he wants that > > py_malloc(0) returns a non-null pointer "normally". > > And there's no better way to be 100% clear about what that means than to > specify a trivial implementation that satisfies it. There could be, but perhaps there isn't. Such an implementation is an overspecification: If a platform already guarantees that malloc(0) acts as if it was passed a non-zero argument, there is no need to pass 1; passing 1 may have undesirable side effects. Regards, Martin From aahz@pythoncraft.com Fri Apr 5 18:25:02 2002 From: aahz@pythoncraft.com (Aahz) Date: Fri, 5 Apr 2002 13:25:02 -0500 Subject: [Python-Dev] Pymalloc and backward compatibility In-Reply-To: References: <20020405174455.GA14217@panix.com> Message-ID: <20020405182501.GA24136@panix.com> On Fri, Apr 05, 2002, Tim Peters wrote: > [Aahz] >> ... >> I am not talking about some abstract general case; it looks to me that in >> this specific case backward compatibility isn't an issue, and I still >> have not seen any explanation for why I'm wrong. > > Well, give us a reason to believe you're right . Specific cases can't > be decided on "general principles" -- the only way to know what vim's needs > actually are is to study its source code. Have you done that? I haven't. You'll note that I started this exchange by asking why pre-2.0 was a requirement, with none of the responses directly addressing my question. I'm certainly not competent to analyze the code, but I think I can at least ask interesting questions. ;-) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From martin@v.loewis.de Fri Apr 5 18:27:58 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 05 Apr 2002 20:27:58 +0200 Subject: [Python-Dev] Pymalloc and backward compatibility In-Reply-To: <20020405172548.GA8114@panix.com> References: <714DFA46B9BBD0119CD000805FC1F53B01B5B2A4@UKRUX002.rundc.uk.origin-it.com> <20020405151519.GA4682@panix.com> <200204051721.g35HL7P14646@pcp742651pcs.reston01.va.comcast.net> <20020405172548.GA8114@panix.com> Message-ID: Aahz writes: > I must be missing something here. We're not talking > about an end-user product, we're talking about a developer tool. What's > wrong with requiring a Python download for someone who at this point > wants to *upgrade* vim from the system default? Because requiring this software might violate company policies. For example, Redhat cannot include more recent versions of Python in their distribution 7.x. More precisely, they can (and do), but they cannot have other packages depend on Python 2.x. They might be able to upgrade vim versions, though (provided those were compatible with the one included in 7.0). If more recent vim versions require more recent Python versions, they cannot upgrade vim. Now, if package xyz required a more recent vim version, they couldn't upgrade that package, either. Regards, Martin From fdrake@acm.org Fri Apr 5 18:30:58 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 5 Apr 2002 13:30:58 -0500 Subject: [Python-Dev] Searching Python docs In-Reply-To: <03e401c1dcce$bdb7fd50$e000a8c0@thomasnotebook> References: <3CAC2F2B.1950AD8F@lemburg.com> <018e01c1dbee$51b1c9f0$e000a8c0@thomasnotebook> <15532.48440.566653.321044@magrathea.basistech.com> <025701c1dcb3$037af110$e000a8c0@thomasnotebook> <3CADD457.4D821311@lemburg.com> <03e401c1dcce$bdb7fd50$e000a8c0@thomasnotebook> Message-ID: <15533.60770.80493.19497@grendel.zope.com> [This is cross-posted; followups to the Doc-SIG only, please!] Thomas Heller writes: > Hm, in the good old days this was the only way ;-) > because the chances where at most 50% that this stuff > was undocumented. I think a lot of us here remember when that 50% was less than 5%, and there was no C API manual. ;-) I still remember seeing the checkin message for api.tex 1.1. > But the docs have improved in the meantime, now you > could also use *them*. Which brings me to the issue of a decent index. (What doesn't, eventually?) We currently have two documents that discuss the C API, though they take (mostly) different approaches. The API reference has a tolerable index (the symbols are there, at least), but the Extending & Embedding document doesn't have enough index entries to be very useful (so little we don't even include an index), even though it contains some strongly overlapping information. I *think* it might be a good idea to merge the two documents, but I'm not certain I really like that. There is a strong need to add good index entries to the existing Extending & Embedding document at the very least, and a combined index would be very valuable. This is something that's been requested for the entire set of documents on many occaisions, and would be very handy. (Especially if we provided a better interface for it!) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From guido@python.org Fri Apr 5 19:17:03 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 14:17:03 -0500 Subject: [Python-Dev] Searching Python docs In-Reply-To: Your message of "Fri, 05 Apr 2002 13:30:58 EST." <15533.60770.80493.19497@grendel.zope.com> References: <3CAC2F2B.1950AD8F@lemburg.com> <018e01c1dbee$51b1c9f0$e000a8c0@thomasnotebook> <15532.48440.566653.321044@magrathea.basistech.com> <025701c1dcb3$037af110$e000a8c0@thomasnotebook> <3CADD457.4D821311@lemburg.com> <03e401c1dcce$bdb7fd50$e000a8c0@thomasnotebook> <15533.60770.80493.19497@grendel.zope.com> Message-ID: <200204051917.g35JH3w15377@pcp742651pcs.reston01.va.comcast.net> > We currently have two documents that discuss the C API, though they > take (mostly) different approaches. The API reference has a tolerable > index (the symbols are there, at least), but the Extending & Embedding > document doesn't have enough index entries to be very useful (so > little we don't even include an index), even though it contains some > strongly overlapping information. > > I *think* it might be a good idea to merge the two documents, but I'm > not certain I really like that. There is a strong need to add good > index entries to the existing Extending & Embedding document at the > very least, and a combined index would be very valuable. This is > something that's been requested for the entire set of documents on > many occaisions, and would be very handy. (Especially if we provided > a better interface for it!) The E&E document should be turned into a tutorial, with a bunch of platform specific things about building. The API reference documentation (in particular PyArg_ArgParse and Py_BuildValue) should be moved from the E&E document to the API document. --Guido van Rossum (home page: http://www.python.org/~guido/) From martin@v.loewis.de Fri Apr 5 19:36:21 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 05 Apr 2002 21:36:21 +0200 Subject: [Python-Dev] Updating to autoconf 2.5x In-Reply-To: <3CADDFA8.88AB3293@lemburg.com> References: <200204041547.g34Flflv007093@paros.informatik.hu-berlin.de> <15532.31123.535297.863287@12-248-41-177.client.attbi.com> <3CAC8141.83A323C4@lemburg.com> <200204041727.g34HRIR05506@odiug.zope.com> <3CADDFA8.88AB3293@lemburg.com> Message-ID: "M.-A. Lemburg" writes: > BTW, the man-page for ld mentions an optimization option > -On -- does anyone know what the affect of this option > is and whether its GNU specific ? Yes, it is GNU specific. It currently (binutils 2.11) only supports one level of optimization, and that is only used for ELF targets. On ELF targets, if optimization is requested, the optimal table size for the symbol table is computed. Since ELF uses a hashing mechanism, the number of hash collisions depends on the table size. Under optimization, the linker tries all table sizes, and choses the one with the least collisions. Regards, Martin From mal@lemburg.com Fri Apr 5 20:25:27 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 05 Apr 2002 22:25:27 +0200 Subject: [Python-Dev] Updating to autoconf 2.5x References: <200204041547.g34Flflv007093@paros.informatik.hu-berlin.de> <15532.31123.535297.863287@12-248-41-177.client.attbi.com> <3CAC8141.83A323C4@lemburg.com> <200204041727.g34HRIR05506@odiug.zope.com> <3CADDFA8.88AB3293@lemburg.com> Message-ID: <3CAE0837.482AE15@lemburg.com> "Martin v. Loewis" wrote: > > "M.-A. Lemburg" writes: > > > BTW, the man-page for ld mentions an optimization option > > -On -- does anyone know what the affect of this option > > is and whether its GNU specific ? > > Yes, it is GNU specific. It currently (binutils 2.11) only supports > one level of optimization, and that is only used for ELF targets. On > ELF targets, if optimization is requested, the optimal table size for > the symbol table is computed. Since ELF uses a hashing mechanism, the > number of hash collisions depends on the table size. Under > optimization, the linker tries all table sizes, and choses the one > with the least collisions. Thanks for the explanations. Probably not worth looking at for Python, right ? -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From xscottg@yahoo.com Fri Apr 5 20:30:29 2002 From: xscottg@yahoo.com (Scott Gilbert) Date: Fri, 5 Apr 2002 12:30:29 -0800 (PST) Subject: [Python-Dev] Array Enhancements Message-ID: <20020405203029.19286.qmail@web12903.mail.yahoo.com> I noticed that in the newsgroup that someone suggested adding the new boolean type to the array module. I was pleased to see that an appropriate patch adding this feature would be accepted. I would find this useful, and since I'm on the verge of writing (yet another) array module, I'm curious if any of the following would be accepted enhancements. Translation: Since none of the existing array modules that I'm aware of (array, Numeric, Numarray) meet all of my needs, I'd be happy to submit a patch for arraymodule.c if I can make enough changes to meet all of my needs. Otherwise, I'll have to write an internal 'xarray' module for my company to get stuff done with. I doubt this warrants a PEP, so I humbly propose: *** Adding a new typecode 't' to implement a bit array. Implementation would be an array of bytes, but it would be 1 bit per element. 't' is for 'truth value' since 'b' for 'boolean' was already taken by 'b' for 'byte'. Accepted values would be 0, 1, True, False. Looking at the arraymodule.c, this seems like the most work out of any of theses suggestions because all of the other types are easily addressable. Slicing operations are going to be tricky to do quickly and correctly. It was already acknowledged by GvR that this would be desirable. *** Adding pickle support. This is a no brainer I think. Currently we have to call tostring() if we want to serialize the data. *** Changing the 'l' and 'L' typecodes to use LONG_LONG. There isn't a consistent way to get an 8 byte integer out of the array module. About half of our machines at work are Alphas where long is 8 bytes, and the rest are Sparcs and x86's where long is 4 bytes. *** I'd really like it if the array module gave a "hard commitment" to the sizes of the elements instead of just sayings "at least n bytes". None of the other array modules do this either. I know Python has been ported to a bazillion platforms, but what are the exceptions to 'char' being 8 bits, 'short' being a 16 bits, 'int' being 32 bits, 'long long' or __int64 being 64 bits, 'float' being 32 bits, and 'double' being 64 bits? I know that an int is 16 bits on Win16, but does Python live there any more? Even so, there is a 32 bit int type on Win16 as well. I guess changing the docs to give a "hard commitment" to this isn't such a big deal to me personally, because the above are true for every platform I think I'll need this for (alpha, x86, sparc, mips). *** In the absence of fixing the 'l' and 'L' types, adding new typecodes ('n' and 'N' perhaps) that do use LONG_LONG. This seems more backwards compatible, but all it really does is make the 'l' and 'L' typecodes duplicate either 'i' or 'n' depending on the platform specific sizeof(long). In otherwords, if an 'n' typecode was added, who would want to use the 'l' one? I suppose someone who knew they wanted a platform specific long. *** I really need complex types. And more than the functionality provided by Numeric/Numarray, I need complex integer types. We frequently read hardware that gives us complex 16 or 32 bit integers, and there are times when we would use 32 or 64 bit fixed point complex numbers. Sometimes we scale our "soft decision" data so that it would fit fine in a complex 8 bit integer. This could be easily added in one of two ways: either adding a 'z' prefix to the existing typecodes, or by creating new typecodes like such: 'u' - complex bytes (8 bit) 'v' - complex shorts (16 bit) 'w' - complex ints (32 bit) 'x' - complex LONG_LONGs (64 bit) 'y' - complex floats (32 bits) 'z' - complex doubles (64 bits) The downside to a 'z' prefix is that typecodes could now be 2 characters 'zi', 'zb', and that would be a bigger change to the implementation. It's also silly to have complex unsigned types (who wants complex numbers that are only in one quadrant?). The downside to adding 'u', 'v', 'w', 'x', 'y', 'z' is that they aren't very mnemonic, and the namespace for typecodes is getting pretty big. Also, I'm unsure how to get the elements in and out of the typecode for 'x' above (a 2-tuple of PyInt/PyLongs?). Python's complex type is sufficient to hold the others without losing precision. *** The ability to construct an array object from an existing C pointer. We get our memory in all kinds of ways (valloc for page aligned DMA transfers, shmem etc...), and it would be nice not to copy in and copy out in some cases. *** Adding an additional signature to the array constructor that specifies a length instead of initial values. a = array.array('d', [1, 2, 3]) would work as it currently does, but a = array.array('d', 30000) would create an array of 30000 doubles. My current hack to accomplish this is to create a small array and use the Sequence operation * to create an array of the size I really want: a = array.array('d', [0])*300000 Besides creating a (small) temporary array, this calls memcpy 300000 times. Yuk. *** In the absence of the last one, adding a new constructor: a = array.xarray('d', 30000) would create an array of 30000 doubles. *** If a signature for creating large arrays is accepted, an optional keyword parameter to specify the value of each element: a = array.xarray('d', 30000, val=1.0) The default val would be None indicating not to waste time initializing the array. *** Multi-dimensional arrays would be valuable too, but this might be treading too much in the territory of the Numarray guys. (I really wish there was a completed one size fits all of my needs array module.) I would propose the following for multi-dimensional arrays: a = array.array('d', 20000, 20000) or: a = array.xarray('d', 20000, 20000) Well if someone authoritative tells me that all of the above is a great idea, I'll start working on a patch and scratch my plans to create a "not in house" xarray module. Cheers, -Scott Gilbert __________________________________________________ Do You Yahoo!? Yahoo! Tax Center - online filing with TurboTax http://taxes.yahoo.com/ From tim.one@comcast.net Fri Apr 5 20:58:12 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 05 Apr 2002 15:58:12 -0500 Subject: [Python-Dev] Array Enhancements In-Reply-To: <20020405203029.19286.qmail@web12903.mail.yahoo.com> Message-ID: Sounds like a PEP to me. Very breifly: > *** Changing the 'l' and 'L' typecodes to use LONG_LONG. Please follow the lead of the struct module, which now uses 'q' and 'Q' for 8-byte ints. > ... > *** I'd really like it if the array module gave a "hard commitment" to > the sizes of the elements instead of just sayings "at least n bytes". It cannot, unless you abandon the notion that it stores native C types. Even C99 refused to guarantee any "exactly n bytes" types, beyond the trivial sizeof(char)==1 (which is definitional in C: if you need to store a char in 128 bits, that's "one byte" to C). > ... > *** I really need complex types. And more than the functionality > provided by Numeric/Numarray, I need complex integer types. This will meet resistance, as it's a pile of code of no conceivable use to the vast majority of Python users. That is, "code bloat". Instead the array type should be subclassable, and extreme special-purpose hair like "complex integers" should be supplied by extension modules. > ... > *** The ability to construct an array object from an existing C > pointer. We get our memory in all kinds of ways (valloc for page > aligned DMA transfers, shmem etc...), and it would be nice not to copy > in and copy out in some cases. Exploit the buffer interface (but write a PEP so there's a single place to record all this stuff -- I'm not getting sucked into a massive one-shot thread about this ). > ... > Well if someone authoritative tells me that all of the above is a great > idea, At least some of it is at least a good idea . > I'll start working on a patch and scratch my plans to create a > "not in house" xarray module. Please write a PEP as the first step: a design document will be invaluable before this is done. I suggest thinking harder about an extensible design than about piling all conceivable features into it at the start. From guido@python.org Fri Apr 5 21:21:56 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 16:21:56 -0500 Subject: [Python-Dev] Array Enhancements In-Reply-To: Your message of "Fri, 05 Apr 2002 12:30:29 PST." <20020405203029.19286.qmail@web12903.mail.yahoo.com> References: <20020405203029.19286.qmail@web12903.mail.yahoo.com> Message-ID: <200204052121.g35LLut20125@pcp742651pcs.reston01.va.comcast.net> > *** Adding a new typecode 't' to implement a bit array. Implementation > would be an array of bytes, but it would be 1 bit per element. 't' is > for 'truth value' since 'b' for 'boolean' was already taken by 'b' for > 'byte'. Accepted values would be 0, 1, True, False. Looking at the > arraymodule.c, this seems like the most work out of any of theses > suggestions because all of the other types are easily addressable. > Slicing operations are going to be tricky to do quickly and correctly. > It was already acknowledged by GvR that this would be desirable. And I'll repeat: I doubt any of the Python developers has time to implement this, but we'll gladly take patches. (Of course, if it takes a lot of work to punch the patches into shape, we'll gladly sit on them forever -- so it's up to you to make the patches work. :-) Be sure to use the latest CVS. > *** Adding pickle support. This is a no brainer I think. Currently we > have to call tostring() if we want to serialize the data. The array module was designed to allow *efficient* reading and writing of array data from/to files, using the fromfile() and tofile() methods. It will be hard to beat these. But maybe you can use them as part of the pickling. It would be nice if a pickled array was unpicklable by previous Python versions, but that may be too slow. E.g. here's a way to pickle arrays today: import pickle import copy_reg import array def reduce_array(a): return array.array, (a.typecode, a.tolist()) copy_reg.pickle(type(array.array('i')), reduce_array, array.array) a = array.array('i', range(10)) print a s = pickle.dumps(a) b = pickle.loads(s) print b print b == a but that's very slow because it first converts the array to a list and which is then pickled. For large arrays this takes too much space to consider. You'll have to consider: is it important to be able to read pickled arrays on previous Python releases, or it that not a requirement? If it's not, you should probably add a new pickle code for pickled arrays, and do an implementation that writes; - the pickle code (1 char) - the array typecode (1 char) - the array length (is 4 bytes enough?) - the array itemsize (some typecodes aren't the same size across platforms) - the array data (length * itemsize bytes) (Please don't bother making a different version for non-binary pickles.) But you'll also have to consider byte ordering and other cross-platform issues -- pickles are supposed to be 100% cross platform portable! > *** Changing the 'l' and 'L' typecodes to use LONG_LONG. There isn't a > consistent way to get an 8 byte integer out of the array module. About > half of our machines at work are Alphas where long is 8 bytes, and the > rest are Sparcs and x86's where long is 4 bytes. I recommend using a new typecode instead -- changing an existing typecode will break existing code. > *** I'd really like it if the array module gave a "hard commitment" to > the sizes of the elements instead of just sayings "at least n bytes". > None of the other array modules do this either. I know Python has been > ported to a bazillion platforms, but what are the exceptions to 'char' > being 8 bits, 'short' being a 16 bits, 'int' being 32 bits, 'long long' > or __int64 being 64 bits, 'float' being 32 bits, and 'double' being 64 > bits? I know that an int is 16 bits on Win16, but does Python live > there any more? Even so, there is a 32 bit int type on Win16 as well. > > I guess changing the docs to give a "hard commitment" to this isn't > such a big deal to me personally, because the above are true for every > platform I think I'll need this for (alpha, x86, sparc, mips). It's hard to make such a commitment without complicating the code, since C doesn't make hard commitments. What do we do on a platform where there simply isn't a 16-bit integer type? Anyway, why do you need this? If it's in the context of pickling, maybe we can define pickling of arrays as only pickling the minimum guaranteed data width. (But then I'd like to get a warning when I'm pickling an array that contains out-of-bounds values, on platforms where the internal item width doesn't match the external width.) > *** In the absence of fixing the 'l' and 'L' types, adding new > typecodes ('n' and 'N' perhaps) that do use LONG_LONG. This seems more > backwards compatible, but all it really does is make the 'l' and 'L' > typecodes duplicate either 'i' or 'n' depending on the platform > specific sizeof(long). In otherwords, if an 'n' typecode was added, > who would want to use the 'l' one? I suppose someone who knew they > wanted a platform specific long. For example, when they have companion C code that interprets the array data as an array of C longs. > *** I really need complex types. And more than the functionality > provided by Numeric/Numarray, I need complex integer types. We > frequently read hardware that gives us complex 16 or 32 bit integers, > and there are times when we would use 32 or 64 bit fixed point complex > numbers. Sometimes we scale our "soft decision" data so that it would > fit fine in a complex 8 bit integer. This could be easily added in one > of two ways: either adding a 'z' prefix to the existing typecodes, or > by creating new typecodes like such: > > 'u' - complex bytes (8 bit) Ehm, 'u' is already taken (Unicode). > 'v' - complex shorts (16 bit) > 'w' - complex ints (32 bit) > 'x' - complex LONG_LONGs (64 bit) > 'y' - complex floats (32 bits) > 'z' - complex doubles (64 bits) > > The downside to a 'z' prefix is that typecodes could now be 2 > characters 'zi', 'zb', and that would be a bigger change to the > implementation. It's also silly to have complex unsigned types (who > wants complex numbers that are only in one quadrant?). (Beats me. But then, I don't have any use for complex numbers myself. And why would anyone want to use complex ints? :-) > The downside to adding 'u', 'v', 'w', 'x', 'y', 'z' is that they aren't > very mnemonic, and the namespace for typecodes is getting pretty big. The user-visible typecode could be a string, and the internal type code could be something with the high bit set or whatever. Then you'd have localized changes. > Also, I'm unsure how to get the elements in and out of the typecode for > 'x' above (a 2-tuple of PyInt/PyLongs?). Python's complex type is > sufficient to hold the others without losing precision. Or use a new custom class that behaves more like a complex number. At least for input, you should also accept regular complexes. > *** The ability to construct an array object from an existing C > pointer. We get our memory in all kinds of ways (valloc for page > aligned DMA transfers, shmem etc...), and it would be nice not to copy > in and copy out in some cases. But then you get into ownership issues. Who owns that memory? Who can free it? What if someone calls a method on the array that requires the memory to be resized? But it's a useful thing to be able to do, I agree, and it shouldn't be too hard to add a flag that says "I don't own this memory" -- which would mean that the buffer can't be resized at all. > *** Adding an additional signature to the array constructor that > specifies a length instead of initial values. > > a = array.array('d', [1, 2, 3]) > > would work as it currently does, but > > a = array.array('d', 30000) > > would create an array of 30000 doubles. My current hack to accomplish > this is to create a small array and use the Sequence operation * to > create an array of the size I really want: > > a = array.array('d', [0])*300000 > > Besides creating a (small) temporary array, this calls memcpy 300000 > times. Yuk. Yuck indeed, and I've often wanted this myself. That could be a simple patch. > *** In the absence of the last one, adding a new constructor: > > a = array.xarray('d', 30000) > > would create an array of 30000 doubles. Nah, just overload the constructor. > *** If a signature for creating large arrays is accepted, an optional > keyword parameter to specify the value of each element: > > a = array.xarray('d', 30000, val=1.0) > > The default val would be None indicating not to waste time initializing > the array. Sure, except I'm not sure if it's worth leaving the memory uninitialized; this is entirely unheard of in Python. (Except when using the C constructor and an existing buffer, of course.) > *** Multi-dimensional arrays would be valuable too, but this might be > treading too much in the territory of the Numarray guys. Yeah, and it would be a major change in the array module implementation. > (I really wish there was a completed one size fits all of my needs > array module.) Since arrays are all about compromises that trade flexibility for speed and memory footprint, you can't have a one size fits all. :-) > I would propose the following for multi-dimensional arrays: > > a = array.array('d', 20000, 20000) > > or: > > a = array.xarray('d', 20000, 20000) I just realized that multi-dimensional __getitem__ shouldn't be a big deal. The question is, given the above declaration, what a[0] should return: the same as a[0, 0] or a copy of a[0, 0:20000] or a reference to a[0, 0:20000]. > Well if someone authoritative tells me that all of the above is a > great idea, I'll start working on a patch and scratch my plans to > create a "not in house" xarray module. It all depends on the quality of the patch. By the time you're done you may have completely rewritten the array module, and then the question is, wouldn't your own xarray module have been quicker to implement, because it doesn't need to preserve backwards compatibility? --Guido van Rossum (home page: http://www.python.org/~guido/) From martin@v.loewis.de Fri Apr 5 21:23:22 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 05 Apr 2002 23:23:22 +0200 Subject: [Python-Dev] Updating to autoconf 2.5x In-Reply-To: <3CAE0837.482AE15@lemburg.com> References: <200204041547.g34Flflv007093@paros.informatik.hu-berlin.de> <15532.31123.535297.863287@12-248-41-177.client.attbi.com> <3CAC8141.83A323C4@lemburg.com> <200204041727.g34HRIR05506@odiug.zope.com> <3CADDFA8.88AB3293@lemburg.com> <3CAE0837.482AE15@lemburg.com> Message-ID: "M.-A. Lemburg" writes: > Thanks for the explanations. Probably not worth looking at for > Python, right ? Right. Martin From eric@enthought.com Fri Apr 5 20:31:55 2002 From: eric@enthought.com (eric) Date: Fri, 5 Apr 2002 15:31:55 -0500 Subject: [Python-Dev] Array Enhancements References: <20020405203029.19286.qmail@web12903.mail.yahoo.com> Message-ID: <134b01c1dce0$edc2b4b0$6b01a8c0@ericlaptop> Hey Scott, You should consider taking this proposal to the Numeric discussion list. numpy-discussion@lists.sourceforge.net Numarray is still in the formative stages (though already pretty far along), and some of your suggestions might make it into the specification. Multi-dimensional arrays for numeric calculations will (hopefully) make it into the core at some point. Certainly numarray is a candidate for this. > Translation: Since none of the existing array modules that I'm aware of > (array, Numeric, Numarray) meet all of my needs, I'd be happy to submit > a patch for arraymodule.c if I can make enough changes to meet all of > my needs. Otherwise, I'll have to write an internal 'xarray' module > for my company to get stuff done with. If your needs are general purpose enough to fit in the core, they should definitely be discussed for additions to numarray. > > I doubt this warrants a PEP, so I humbly propose: > > > *** Adding a new typecode 't' to implement a bit array. Implementation > would be an array of bytes, but it would be 1 bit per element. 't' is > for 'truth value' since 'b' for 'boolean' was already taken by 'b' for > 'byte'. Accepted values would be 0, 1, True, False. Looking at the > arraymodule.c, this seems like the most work out of any of theses > suggestions because all of the other types are easily addressable. > Slicing operations are going to be tricky to do quickly and correctly. > It was already acknowledged by GvR that this would be desirable. A bit type for numarray has been discussed at least briefly. The implementation for a 1 bit per element array is tricky and hasn't been to high on anyone's implementation list. If you were to propose this and offer to implement it, the Space Telescope guys might accept it. It definitely needs discussion though. > > *** Adding pickle support. This is a no brainer I think. Currently we > have to call tostring() if we want to serialize the data. > Numeric and (I think) numarray are picklable already. If numarray isn't now, it will be I'm sure. > > *** Changing the 'l' and 'L' typecodes to use LONG_LONG. There isn't a > consistent way to get an 8 byte integer out of the array module. About > half of our machines at work are Alphas where long is 8 bytes, and the > rest are Sparcs and x86's where long is 4 bytes. Haven't used LONG_LONG before, but it sounds like this warrants discussion. > > *** I'd really like it if the array module gave a "hard commitment" to > the sizes of the elements instead of just sayings "at least n bytes". > None of the other array modules do this either. I know Python has been > ported to a bazillion platforms, but what are the exceptions to 'char' > being 8 bits, 'short' being a 16 bits, 'int' being 32 bits, 'long long' > or __int64 being 64 bits, 'float' being 32 bits, and 'double' being 64 > bits? I know that an int is 16 bits on Win16, but does Python live > there any more? Even so, there is a 32 bit int type on Win16 as well. > > I guess changing the docs to give a "hard commitment" to this isn't > such a big deal to me personally, because the above are true for every > platform I think I'll need this for (alpha, x86, sparc, mips). > I don't see this one as that big a deal. As you say, most modern platforms treat them the same way. > *** In the absence of fixing the 'l' and 'L' types, adding new > typecodes ('n' and 'N' perhaps) that do use LONG_LONG. This seems more > backwards compatible, but all it really does is make the 'l' and 'L' > typecodes duplicate either 'i' or 'n' depending on the platform > specific sizeof(long). In otherwords, if an 'n' typecode was added, > who would want to use the 'l' one? I suppose someone who knew they > wanted a platform specific long. > > > *** I really need complex types. And more than the functionality > provided by Numeric/Numarray, I need complex integer types. We > frequently read hardware that gives us complex 16 or 32 bit integers, > and there are times when we would use 32 or 64 bit fixed point complex > numbers. Sometimes we scale our "soft decision" data so that it would > fit fine in a complex 8 bit integer. This could be easily added in one > of two ways: either adding a 'z' prefix to the existing typecodes, or > by creating new typecodes like such: I hadn't thought about needing integer complex numbers. It seems possible to add this for numarray. Numeric uses uppercase letters to represent complex versions of numbers ('f' for floats and 'F' for complex floats). The same convention could be used for complex integer types with the exception of Int8 whose character is '1'. > > 'u' - complex bytes (8 bit) > 'v' - complex shorts (16 bit) > 'w' - complex ints (32 bit) > 'x' - complex LONG_LONGs (64 bit) > 'y' - complex floats (32 bits) > 'z' - complex doubles (64 bits) > > The downside to a 'z' prefix is that typecodes could now be 2 > characters 'zi', 'zb', and that would be a bigger change to the > implementation. It's also silly to have complex unsigned types (who > wants complex numbers that are only in one quadrant?). > > The downside to adding 'u', 'v', 'w', 'x', 'y', 'z' is that they aren't > very mnemonic, and the namespace for typecodes is getting pretty big. > > Also, I'm unsure how to get the elements in and out of the typecode for > 'x' above (a 2-tuple of PyInt/PyLongs?). Python's complex type is > sufficient to hold the others without losing precision. > > > *** The ability to construct an array object from an existing C > pointer. We get our memory in all kinds of ways (valloc for page > aligned DMA transfers, shmem etc...), and it would be nice not to copy > in and copy out in some cases. > > > > *** Adding an additional signature to the array constructor that > specifies a length instead of initial values. > > a = array.array('d', [1, 2, 3]) > > would work as it currently does, but > > a = array.array('d', 30000) > > would create an array of 30000 doubles. My current hack to accomplish > this is to create a small array and use the Sequence operation * to > create an array of the size I really want: > > a = array.array('d', [0])*300000 > > Besides creating a (small) temporary array, this calls memcpy 300000 > times. Yuk. > In Numeric, zeros(30000,typecode='d') or ones(30000,typecode='f') work for doing this. > > > *** In the absence of the last one, adding a new constructor: > > a = array.xarray('d', 30000) > > would create an array of 30000 doubles. > > > > *** If a signature for creating large arrays is accepted, an optional > keyword parameter to specify the value of each element: > > a = array.xarray('d', 30000, val=1.0) > > The default val would be None indicating not to waste time initializing > the array. > Initializing to a specific value can be done with: ones(30000,typecode='f') * value This does require a multiplication, so it isn't as fast as your proposal. Adding your proposed function to Numeric is minimal effort -- it may even be lurking there now, though I have never seen or used it. > > > *** Multi-dimensional arrays would be valuable too, but this might be > treading too much in the territory of the Numarray guys. (I really > wish there was a completed one size fits all of my needs array module.) > I would propose the following for multi-dimensional arrays: > > a = array.array('d', 20000, 20000) > > or: > > a = array.xarray('d', 20000, 20000) > Any multi-dimensional array really should come from the conventions used by Numeric. It sounds like the missing features from Numarray that your interested in are: bit arrays, support for LONG_LONG, and support for complex integers. The other features are already in Numeric/numarray -- perhaps with syntax. Is this the full picture? All seem reasonably general and worth discussing for numarray. By the way, numarray is written so that it can be sub-classed. This means you could add the features you want in a derived class if the features are accepted into the standard. regards, eric From DavidA@ActiveState.com Fri Apr 5 21:37:23 2002 From: DavidA@ActiveState.com (David Ascher) Date: Fri, 05 Apr 2002 13:37:23 -0800 Subject: [Python-Dev] Array Enhancements References: <20020405203029.19286.qmail@web12903.mail.yahoo.com> <200204052121.g35LLut20125@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CAE1913.ECB27329@activestate.com> Guido van Rossum wrote: > > I would propose the following for multi-dimensional arrays: > > > > a = array.array('d', 20000, 20000) > > > > or: > > > > a = array.xarray('d', 20000, 20000) > > I just realized that multi-dimensional __getitem__ shouldn't be a big > deal. The question is, given the above declaration, what a[0] should > return: the same as a[0, 0] or a copy of a[0, 0:20000] or a reference > to a[0, 0:20000]. Or a ValueError? In the face of ambiguity, refuse the temptation to guess. IIRC, this issue caused lots of problems in the numpy world. cc'ing Paul in case he wants to jump in to fill in my rusty memory. Why does submitting a patch to arraymodule seem an easier path than modifying numarray or numpy to support what's needed? I believe that the goals of numarray aren't that different from what Scott is trying to do (memory management APIs, etc.). I'd like to see fewer multi-dimensional array objects, not more... --david ascher From guido@python.org Fri Apr 5 21:47:52 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 16:47:52 -0500 Subject: [Python-Dev] Array Enhancements In-Reply-To: Your message of "Fri, 05 Apr 2002 15:31:55 EST." <134b01c1dce0$edc2b4b0$6b01a8c0@ericlaptop> References: <20020405203029.19286.qmail@web12903.mail.yahoo.com> <134b01c1dce0$edc2b4b0$6b01a8c0@ericlaptop> Message-ID: <200204052147.g35Llqm20287@pcp742651pcs.reston01.va.comcast.net> > A bit type for numarray has been discussed at least briefly. The > implementation for a 1 bit per element array is tricky and hasn't > been to high on anyone's implementation list. If you were to > propose this and offer to implement it, the Space Telescope guys > might accept it. It definitely needs discussion though. An alternative might be a separate bit-array implementation: it seems that the bit-array won't share much code with the regular array (of any flavor), so why not make it a separate type? In addition, Scott should definitely consider whether Numarray is a better target for his requirements than the core array module. It may very well replace the current array module in the future, so your changes might have a brighter future that way. --Guido van Rossum (home page: http://www.python.org/~guido/) From sjmachin@lexicon.net Fri Apr 5 21:50:35 2002 From: sjmachin@lexicon.net (John Machin) Date: Sat, 06 Apr 2002 07:50:35 +1000 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) In-Reply-To: <025701c1dcb3$037af110$e000a8c0@thomasnotebook> Message-ID: 06/04/2002 1:03:15 AM, "Thomas Heller" wrote: >Anyway, at least on one question I would really like to get more >answers : when writing python code or C extensions, how do people >quickly look up definitions of functions, let's say PyObject_GC_New() >or the meaning of METH_O ? (On Windows) using the .CHM file provided by Hernan Foffani; can copy from another application and paste into the search or index window; with your example, index also shows 7 other functions/macros matching ^_?PyObject_GC -- very handy. Is it a FAQ why this kit is not in the Pythonlabs Win32 distribution instead of the "bunch of HTML files" kit? I understand that it's in the Activestate dist'n. Regards, John From fdrake@acm.org Fri Apr 5 22:04:49 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 5 Apr 2002 17:04:49 -0500 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) In-Reply-To: References: <025701c1dcb3$037af110$e000a8c0@thomasnotebook> Message-ID: <15534.8065.891619.224874@grendel.zope.com> John Machin writes: > Is it a FAQ why this kit is not in the Pythonlabs Win32 distribution > instead of the "bunch of HTML files" kit? I understand that > it's in the Activestate dist'n. It's generally not available when the Windows installer is built. Hernan is quick, but not *that* fast! -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From tim.one@comcast.net Fri Apr 5 22:07:51 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 05 Apr 2002 17:07:51 -0500 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) In-Reply-To: Message-ID: [John Machin] > (On Windows) using the .CHM file provided by Hernan Foffani; > can copy from another application and paste into the search or index > window; with your example, index also shows 7 other functions/macros > matching ^_?PyObject_GC -- very handy. > > Is it a FAQ why this kit is not in the Pythonlabs Win32 distribution > instead of the "bunch of HTML files" kit? Nope -- IIRC, you're the first to ask. > I understand that it's in the Activestate dist'n. I'm not clear on "it's" referrent; I suspect ActiveState rolls its own doc packages. If Fred volunteers to produce .chm files in time for a release, I'll ship 'em. Don't hold your breath . our-support-for-the-windows-distro-is-bug-driven-ly y'rs - tim From perry@stsci.edu Fri Apr 5 22:26:15 2002 From: perry@stsci.edu (Perry Greenfield) Date: Fri, 5 Apr 2002 17:26:15 -0500 Subject: [Python-Dev] Array Enhancements In-Reply-To: <134b01c1dce0$edc2b4b0$6b01a8c0@ericlaptop> Message-ID: > -----Original Message----- > From: eric [mailto:eric@enthought.com] > Sent: Friday, April 05, 2002 3:32 PM > To: python-dev@python.org > Cc: Scott Gilbert; Perry Greenfield; travis oliphant > Subject: Re: [Python-Dev] Array Enhancements > > > Hey Scott, > > You should consider taking this proposal to the Numeric discussion list. > > numpy-discussion@lists.sourceforge.net > > Numarray is still in the formative stages (though already pretty > far along), and > some of your suggestions might make it into the specification. > Multi-dimensional arrays for numeric calculations will > (hopefully) make it into > the core at some point. Certainly numarray is a candidate for this. > > > Translation: Since none of the existing array modules that I'm aware of > > (array, Numeric, Numarray) meet all of my needs, I'd be happy to submit > > a patch for arraymodule.c if I can make enough changes to meet all of > > my needs. Otherwise, I'll have to write an internal 'xarray' module > > for my company to get stuff done with. > > If your needs are general purpose enough to fit in the core, they should > definitely be discussed for additions to numarray. > > > > > I doubt this warrants a PEP, so I humbly propose: > > > > > > *** Adding a new typecode 't' to implement a bit array. Implementation > > would be an array of bytes, but it would be 1 bit per element. 't' is > > for 'truth value' since 'b' for 'boolean' was already taken by 'b' for > > 'byte'. Accepted values would be 0, 1, True, False. Looking at the > > arraymodule.c, this seems like the most work out of any of theses > > suggestions because all of the other types are easily addressable. > > Slicing operations are going to be tricky to do quickly and correctly. > > It was already acknowledged by GvR that this would be desirable. > > A bit type for numarray has been discussed at least briefly. The > implementation > for a 1 bit per element array is tricky and hasn't been to high > on anyone's > implementation list. If you were to propose this and offer to > implement it, the > Space Telescope guys might accept it. It definitely needs > discussion though. > Bit array have definitely been discussed. We aren't opposed to them at all (and could use them ourselves). On the other hand, they aren't high enough priority for us to implement them within the next 6 months (or even year). The problem with bit arrays is that for the most part, they would not use much of the existing numarray code, so it is a moderate amount of work to add if you want it to be speed and memory efficient. But if someone else wants to do the implementation, we would welcome it. (But we don't use typecodes so 't' is right out :-) > > > > *** Adding pickle support. This is a no brainer I think. Currently we > > have to call tostring() if we want to serialize the data. > > > > Numeric and (I think) numarray are picklable already. If > numarray isn't now, it > will be I'm sure. > We just got a question about this. numarray doesn't currently support pickling. But it will. We have begun to think about how to best implement it. > > > > *** Changing the 'l' and 'L' typecodes to use LONG_LONG. There isn't a > > consistent way to get an 8 byte integer out of the array module. About > > half of our machines at work are Alphas where long is 8 bytes, and the > > rest are Sparcs and x86's where long is 4 bytes. > > Haven't used LONG_LONG before, but it sounds like this warrants > discussion. > We are proponents of using types to represent the same sized numeric types on all machines. numarray does not yet support Int64. Ultimately it will, but we need to give some thought to how we deal with nonportable (i.e., not available on all platforms) numeric types. This does warrant discussion. > > > > *** I'd really like it if the array module gave a "hard commitment" to > > the sizes of the elements instead of just sayings "at least n bytes". > > None of the other array modules do this either. I know Python has been > > ported to a bazillion platforms, but what are the exceptions to 'char' > > being 8 bits, 'short' being a 16 bits, 'int' being 32 bits, 'long long' > > or __int64 being 64 bits, 'float' being 32 bits, and 'double' being 64 > > bits? I know that an int is 16 bits on Win16, but does Python live > > there any more? Even so, there is a 32 bit int type on Win16 as well. > > Well, as mentioned, we believe the numarray types should be of definite sizes and we've implemented this so they are. We can't do anything about Python scalars however, nor should we. We have no plans for dealing with Win16. It doesn't exist as far as we are concerned. > > I guess changing the docs to give a "hard commitment" to this isn't > > such a big deal to me personally, because the above are true for every > > platform I think I'll need this for (alpha, x86, sparc, mips). > > > > I don't see this one as that big a deal. As you say, most modern > platforms > treat them the same way. > > > *** In the absence of fixing the 'l' and 'L' types, adding new > > typecodes ('n' and 'N' perhaps) that do use LONG_LONG. This seems more > > backwards compatible, but all it really does is make the 'l' and 'L' > > typecodes duplicate either 'i' or 'n' depending on the platform > > specific sizeof(long). In otherwords, if an 'n' typecode was added, > > who would want to use the 'l' one? I suppose someone who knew they > > wanted a platform specific long. > > > > > > *** I really need complex types. And more than the functionality > > provided by Numeric/Numarray, I need complex integer types. We > > frequently read hardware that gives us complex 16 or 32 bit integers, > > and there are times when we would use 32 or 64 bit fixed point complex > > numbers. Sometimes we scale our "soft decision" data so that it would > > fit fine in a complex 8 bit integer. This could be easily added in one > > of two ways: either adding a 'z' prefix to the existing typecodes, or > > by creating new typecodes like such: > > I hadn't thought about needing integer complex numbers. It seems > possible to > add this for numarray. Numeric uses uppercase letters to > represent complex > versions of numbers ('f' for floats and 'F' for complex floats). The same > convention could be used for complex integer types with the > exception of Int8 > whose character is '1'. > Complex ints? We sure don't need them and I suspect that is a fairly niche type. I would be against making them part of the core numarray. It is possible to add such types to numarray "on the fly" or create subclasses that support them. But there is enough bloat with existing (and potential future common types--e.g., Int64, Int128, Float128...) that I would avoid them as part of the base numarray. > > > > 'u' - complex bytes (8 bit) > > 'v' - complex shorts (16 bit) > > 'w' - complex ints (32 bit) > > 'x' - complex LONG_LONGs (64 bit) > > 'y' - complex floats (32 bits) > > 'z' - complex doubles (64 bits) > > > > The downside to a 'z' prefix is that typecodes could now be 2 > > characters 'zi', 'zb', and that would be a bigger change to the > > implementation. It's also silly to have complex unsigned types (who > > wants complex numbers that are only in one quadrant?). > > > > The downside to adding 'u', 'v', 'w', 'x', 'y', 'z' is that they aren't > > very mnemonic, and the namespace for typecodes is getting pretty big. > > As mentioned, numarray doesn't directly use typecodes (though for backward compatibility it does recognize existing ones). We would like to steer users away from them (they are confusing enough without the proposed additions above!). Since so few of the ufuncs appear to make much sense for integer complex types (transendental functions, most comparison or bit functions...), this is almost certainly better implemented as a subclass of NDArray in numarray (almost how complex is currently implemented though we plan to replace that with a C implementation). > > Also, I'm unsure how to get the elements in and out of the typecode for > > 'x' above (a 2-tuple of PyInt/PyLongs?). Python's complex type is > > sufficient to hold the others without losing precision. > > > > > > *** The ability to construct an array object from an existing C > > pointer. We get our memory in all kinds of ways (valloc for page > > aligned DMA transfers, shmem etc...), and it would be nice not to copy > > in and copy out in some cases. > > In one sense this is simple. But the real issue is how will Python manage memory for these C pointers? If something else deallocates the memory, how do you know that a numeric object or numarray object isn't using it? Without specifying how memory managment is to be done, it isn't possible to use "outside" pointers safely. But maybe I misunderstand. > > > > > > *** Adding an additional signature to the array constructor that > > specifies a length instead of initial values. > > > > a = array.array('d', [1, 2, 3]) > > > > would work as it currently does, but > > > > a = array.array('d', 30000) > > > > would create an array of 30000 doubles. My current hack to accomplish > > this is to create a small array and use the Sequence operation * to > > create an array of the size I really want: > > > > a = array.array('d', [0])*300000 > > > > Besides creating a (small) temporary array, this calls memcpy 300000 > > times. Yuk. > > > > In Numeric, > > zeros(30000,typecode='d') or > ones(30000,typecode='f') > > work for doing this. Yup. > > > > > > *** In the absence of the last one, adding a new constructor: > > > > a = array.xarray('d', 30000) > > > > would create an array of 30000 doubles. > > > > > > > > *** If a signature for creating large arrays is accepted, an optional > > keyword parameter to specify the value of each element: > > > > a = array.xarray('d', 30000, val=1.0) > > > > The default val would be None indicating not to waste time initializing > > the array. > > > Initializing to a specific value can be done with: > > ones(30000,typecode='f') * value > > This does require a multiplication, so it isn't as fast as your proposal. > Adding your proposed function to Numeric is minimal effort -- it > may even be > lurking there now, though I have never seen or used it. > I really doubt that the performance penalty is worth bothering about. If this is one of your biggest performance problems, I'd like to trade :-) But it is trivial to add (if it isn't already in numarray). > > > > > > *** Multi-dimensional arrays would be valuable too, but this might be > > treading too much in the territory of the Numarray guys. (I really > > wish there was a completed one size fits all of my needs array module.) > > I would propose the following for multi-dimensional arrays: > > > > a = array.array('d', 20000, 20000) > > > > or: > > > > a = array.xarray('d', 20000, 20000) > > > or a = zeros((20000,20000),typecode='d') # for Numeric > Any multi-dimensional array really should come from the > conventions used by > Numeric. It sounds like the missing features from Numarray that > your interested > in are: bit arrays, support for LONG_LONG, and support for > complex integers. > The other features are already in Numeric/numarray -- perhaps > with syntax. Is > this the full picture? All seem reasonably general and worth > discussing for > numarray. > > By the way, numarray is written so that it can be sub-classed. > This means you > could add the features you want in a derived class if the > features are accepted > into the standard. > > regards, > eric > > Eric summarizes it correctly. numarraying'ly yours, Perry From trentm@ActiveState.com Fri Apr 5 23:30:12 2002 From: trentm@ActiveState.com (Trent Mick) Date: Fri, 5 Apr 2002 15:30:12 -0800 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) In-Reply-To: ; from tim.one@comcast.net on Fri, Apr 05, 2002 at 05:07:51PM -0500 References: Message-ID: <20020405153012.B16575@ActiveState.com> [Tim Peters wrote] > > I understand that it's in the Activestate dist'n. > > I'm not clear on "it's" referrent; I suspect ActiveState rolls its own doc > packages. Yup, we do. And it is a PITA. :) Trent -- Trent Mick TrentM@ActiveState.com From jeremy@zope.com Fri Apr 5 23:51:48 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Fri, 5 Apr 2002 18:51:48 -0500 Subject: [Python-Dev] Deprecating string exceptions In-Reply-To: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> Message-ID: <15534.14484.134938.791320@slothrop.zope.com> >>>>> "MvL" == Martin v Loewis writes: MvL> http://python.org/sf/518846 reports that new-style classes MvL> cannot be used as exceptions. I think it is desirable that this MvL> is fixed, but I also believe that it conflicts with string MvL> exceptions. So I would like to propose that string exceptions MvL> are deprecated for Python 2.3, in order to remove them in MvL> Python 2.4, simultaneously allowing arbitrary objects as MvL> exceptions. I didn't read most of this length thread the first time through, but I did run into the bug today in my own code and tried to fix it. I added a patch to 518846 that allows instances of new style classes, where the check is implemented by checking Py_TPFLAGS_HEAPTYPE, which is only defined for objects created by class statements (I think). This doesn't seem to cause any change in the behavior of string-based exceptions. I don't think it's necessary to have "except str:" catch string-based exceptions, because they're supported solely for backwards compatibility. There was also some discussion of making exceptions inherit from Exception, but I think that's a separate issue. Any reason not to commit the patch? Jeremy From sjmachin@lexicon.net Sat Apr 6 00:14:00 2002 From: sjmachin@lexicon.net (John Machin) Date: Sat, 06 Apr 2002 10:14:00 +1000 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) In-Reply-To: <15534.8065.891619.224874@grendel.zope.com> Message-ID: [I asked:] > > Is it a FAQ why this kit is not in the Pythonlabs Win32 distribution > > instead of the "bunch of HTML files" kit? I understand that > > it's in the Activestate dist'n. [Fred Drake] >It's generally not available when the Windows installer is built. >Hernan is quick, but not *that* fast! [Tim Peters] If Fred volunteers to produce .chm files in time for a release, I'll ship 'em. Don't hold your breath . our-support-for-the-windows-distro-is-bug-driven-ly y'rs - tim Now read on ... Tim's tag-line might explain it all to me if I could nut outexactly what he meant. However, blundering on in blissful ignorance of the logistics, legalities, real-politik, etc: Premises: (1) .chm format is better for the windows distro than the .html format (2) [not sure why Tim used the plural "files"] One big .chm file is better than multiple smaller ones -- e.g. will find PyArg_BuildValue() without the punter needing to know whether it's in the the E&E manual or in the C API manual :-) (3) A script can be developed [Hernan presumably has at least the foundation for this] to create the .chm file from the .html files so that producing the .chm file at release time requires neither volunteer nor conscript intervention ... And some afterthoughts on Tim's tagline, which in my ignorance I interpreted as indicative of a somewhat reactive attitude to the win32 distro [I'd be utterly delighted to be corrected if I'm wrong]: (1) Judging from the traffic on c.l.py, Python seems to have reached the point in a language's life cycle where it is attracting many novice users. My guess is that most of those will be using the win32 distribution. Many of them will no doubt refuse to RTFM, but for those who will, providing them with the docs in .chm format (and actively promoting it) might cut down on the "support cost" of the traffic on c.l.py. (2) In two application development shops with which I am engaged, the main application runs on a large Unix box, but the developers' and DBAs' workstations are Windows (2000) PCs -- is this atypical? I am evangelising Python to them -- while certainly not critical, better doc formats, positive attitude to win32 distro, etc etc wouldn't hurt. Regards, John From bckfnn@worldonline.dk Sat Apr 6 00:13:55 2002 From: bckfnn@worldonline.dk (Finn Bock) Date: Sat, 06 Apr 2002 00:13:55 GMT Subject: [Python-Dev] Array Enhancements In-Reply-To: <20020405203029.19286.qmail@web12903.mail.yahoo.com> References: <20020405203029.19286.qmail@web12903.mail.yahoo.com> Message-ID: <3cae3ce4.1321550@mail.wanadoo.dk> [Scott Gilbert] >*** Adding a new typecode 't' to implement a bit array. In Jython (and in java JNI) the letter 'z' have been used to specify boolean: http://www.jython.org/docs/jarray.html regards, finn From guido@python.org Sat Apr 6 00:18:19 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 19:18:19 -0500 Subject: [Python-Dev] Deprecating string exceptions In-Reply-To: Your message of "Fri, 05 Apr 2002 18:51:48 EST." <15534.14484.134938.791320@slothrop.zope.com> References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <15534.14484.134938.791320@slothrop.zope.com> Message-ID: <200204060018.g360IKI22119@pcp742651pcs.reston01.va.comcast.net> > I didn't read most of this length thread the first time through, but I > did run into the bug today in my own code and tried to fix it. I > added a patch to 518846 that allows instances of new style classes, > where the check is implemented by checking Py_TPFLAGS_HEAPTYPE, which > is only defined for objects created by class statements (I think). > > This doesn't seem to cause any change in the behavior of string-based > exceptions. I don't think it's necessary to have "except str:" catch > string-based exceptions, because they're supported solely for > backwards compatibility. > > There was also some discussion of making exceptions inherit from > Exception, but I think that's a separate issue. > > Any reason not to commit the patch? In how many places have you posted about this issue? This is the third place where I see you trying to get people's attention. What's the urgency? :-) I don't think the patch is ready yet, see my comment on SF. If we decide that anything needs to be done in this area I want to look at it first, so please don't check anything in yet. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Sat Apr 6 00:33:44 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 19:33:44 -0500 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) In-Reply-To: Your message of "Sat, 06 Apr 2002 10:14:00 +1000." References: Message-ID: <200204060033.g360Xit22223@pcp742651pcs.reston01.va.comcast.net> > (1) .chm format is better for the windows distro than the .html format Speak for yourself. I prefer the HTML that I can view in a browser of my choice any time over MS's clumsy help browser. > And some afterthoughts on Tim's tagline, which in my ignorance I > interpreted as indicative of a somewhat reactive attitude to the > win32 distro [I'd be utterly delighted to be corrected if I'm > wrong]: You're wrong. > (1) Judging from the traffic on c.l.py, > Python seems to have reached the point in a > language's life cycle where it is attracting many novice users. Tell us something we don't know. > My guess is that most of those will be using the win32 > distribution. Many of them will no doubt refuse to RTFM, but for > those who will, providing them with the docs in .chm format (and > actively promoting it) might cut down on the "support cost" of the > traffic on c.l.py. I strongly doubt that the *format* we use to provide the docs makes much of a difference. > (2) In two application development shops with which I am engaged, > the main application runs on a large Unix box, but the developers' > and DBAs' workstations are Windows (2000) PCs -- is this atypical? I've never seen it, but I suppose I'm atypical. :-) > I am evangelising Python to them -- while certainly not critical, > better doc formats, positive attitude to win32 distro, etc etc > wouldn't hurt. Stop complaining and do something about it. If you think Tim's got an negative attitude about win32 distro, I think you need a reality adjustment. --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@pythoncraft.com Sat Apr 6 00:47:06 2002 From: aahz@pythoncraft.com (Aahz) Date: Fri, 5 Apr 2002 19:47:06 -0500 Subject: [Python-Dev] Namespaces In-Reply-To: References: <002901c1d7f2$8b04f560$2eb53bd0@othello> <20020401025823.GA29354@panix.com> <20020401185657.GA6094@panix.com> <15528.51794.642978.22967@slothrop.zope.com> <20020401212232.GA14508@panix.com> Message-ID: <20020406004705.GA751@panix.com> [After going away and thinking about this for a few days...] On Tue, Apr 02, 2002, Martin v. Loewis wrote: > Aahz writes: >> >> I'm fine with "local scope" and "object attributes" to disambiguate >> them; I just think it's important that people understand that a name is >> a name is a name, and all names live in *some* namespace. > > That isn't really true: a computed attribute lives in no namespace, > instead, some function is invoked to determine the attribute value. You're right, though I think I prefer "computed binding" to "computed attribute" (assuming you're referring to the __getitem__ / __setitem__ protocol). The question I'm now wrestling with is what to call bindings in general. I'm tripping over this: Rebinding a does not affect the originally bound object (unless the originally bound object's reference count goes to zero). Any ideas about what to call ? (Calling it a binding sounds a little too self-referential.) > Furthermore, some attributes live in multiple namespaces. Given > > obj.name > > what namespace is considered to find the name? NOT the namespace of > obj, alone - Python also considers the namespace of obj's class (if > obj is an instance), of the base classes, etc. OTOH, > > obj.name = None > > modifies the namespace of obj (unless name is a computed attribute). Exactly. Binding operations are different from binding lookups. I'm not sure I understand your unless, though; can you demonstrate with code? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From jeremy@zope.com Sat Apr 6 01:03:53 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Fri, 5 Apr 2002 20:03:53 -0500 Subject: [Python-Dev] Deprecating string exceptions In-Reply-To: <200204060018.g360IKI22119@pcp742651pcs.reston01.va.comcast.net> References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <15534.14484.134938.791320@slothrop.zope.com> <200204060018.g360IKI22119@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15534.18809.964868.250044@slothrop.zope.com> >>>>> "GvR" == Guido van Rossum writes: >> I didn't read most of this length thread the first time through, >> but I did run into the bug today in my own code and tried to fix >> it. I added a patch to 518846 that allows instances of new style >> classes, where the check is implemented by checking >> Py_TPFLAGS_HEAPTYPE, which is only defined for objects created by >> class statements (I think). >> >> This doesn't seem to cause any change in the behavior of >> string-based exceptions. I don't think it's necessary to have >> "except str:" catch string-based exceptions, because they're >> supported solely for backwards compatibility. >> >> There was also some discussion of making exceptions inherit from >> Exception, but I think that's a separate issue. >> >> Any reason not to commit the patch? GvR> In how many places have you posted about this issue? This is GvR> the third place where I see you trying to get people's GvR> attention. What's the urgency? :-) There's no urgency at all but a bug report isn't the right place for the discussin, nor is our group's private email. I didn't think that raising the question in the appropriate forum constituted emergency action :-). GvR> I don't think the patch is ready yet, see my comment on SF. If GvR> we decide that anything needs to be done in this area I want to GvR> look at it first, so please don't check anything in yet. Did I miss the consensus on inheriting from Exception? I tried to review the earlier thread, but the early stuff didn't have much on Exception and the later stuff seemed to be about documentation. So it must be buried in the middle if it's there :-). At any rate, I don't understand why it is helpful for classes to inherit from exception. I am also curious -- this is off-topic for exceptions -- what the right way to check for a new-style class is? And what other things define Py_TPFLAGS_HEAPTYPE? Jeremy From jeremy@zope.com Sat Apr 6 01:05:55 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Fri, 5 Apr 2002 20:05:55 -0500 Subject: [Python-Dev] Namespaces In-Reply-To: <20020406004705.GA751@panix.com> References: <002901c1d7f2$8b04f560$2eb53bd0@othello> <20020401025823.GA29354@panix.com> <20020401185657.GA6094@panix.com> <15528.51794.642978.22967@slothrop.zope.com> <20020401212232.GA14508@panix.com> <20020406004705.GA751@panix.com> Message-ID: <15534.18931.480692.612031@slothrop.zope.com> I don't think that setting or getting an attribute is a name binding operation. Jeremy From David Abrahams" <200204060033.g360Xit22223@pcp742651pcs.reston01.va.comcast.net> Message-ID: <02f701c1dd08$cecea2d0$6401a8c0@boostconsulting.com> ----- Original Message ----- From: "Guido van Rossum" > > My guess is that most of those will be using the win32 > > distribution. Many of them will no doubt refuse to RTFM, but for > > those who will, providing them with the docs in .chm format (and > > actively promoting it) might cut down on the "support cost" of the > > traffic on c.l.py. > > I strongly doubt that the *format* we use to provide the docs makes > much of a difference. *Strongly* disagree. As someone who doesn't know where everything is using docs where many things are in the wrong place, having built-in fast searching is absolutely essential for my work. I can just imagine how it is for a novice... as someone who's got the whole world in his brain, Guido, I think you should consider disqualifying yourself from trying to intuit what people need in the documentation department. -Dave From tim.one@comcast.net Sat Apr 6 01:34:58 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 05 Apr 2002 20:34:58 -0500 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) In-Reply-To: Message-ID: [Tim] >> If Fred volunteers to produce .chm files in time for a release, >> I'll ship 'em. Don't hold your breath . >> our-support-for-the-windows-distro-is-bug-driven-ly y'rs - tim [John Machin] > Tim's tag-line might explain it all to me if I could nut > outexactly what he meant. I mean that if you look at any change to the PLabs Windows installer checked in since I started doing it, it's in response to a bug, or to a change in the structure of Python components (files moving around, getting added, getting removed). IIRC, the only exception (== "new feature" not addressing a use problem) was to add an "Edit with IDLE" context menu entry for .py files, requested by Guido. > However, blundering on in blissful ignorance of the logistics, > legalities, real-politik, etc: > > Premises: > (1) .chm format is better for the windows distro than the .html format I prefer HTML myself, but I confess I rarely need to *search* for things in the docs (and when I do, I grep over the raw .tex files -- not a realistic option for my sisters ). > (2) [not sure why Tim used the plural "files"] One big .chm file > is better than multiple smaller ones -- e.g. will find > PyArg_BuildValue() without the punter needing to know whether it's > in the the E&E manual or in the C API manual :-) Sure. Packing over 1,000 HTML files into the installer bloats it too (it uses zip format, and doesn't appear to exploit opportunities for x-file compression). > (3) A script can be developed [Hernan presumably has at least the > foundation for this] to create the .chm file from the .html files > so that producing the .chm file at release time requires neither > volunteer nor conscript intervention ... Possibly, but the idea that a new software process involving many megabytes of data won't create new time sinks isn't credible to me. Fred takes pride in the appearance of his docs, so won't want to let small points slide either. > And some afterthoughts on Tim's tagline, which in my ignorance I > interpreted as indicative of a somewhat reactive attitude to the win32 > distro [I'd be utterly delighted to be corrected if I'm wrong]: All work on the Windows installer comes out of "bug fixing" time or out of my sleep. It's not purely reactive, but there are no development plans for it, and, e.g., I doubt PLabs will ever produce an MSI installer. ActiveState's installers have been better for some time already, and I expect the gap to widen. All the work PLabs can do here is charity work (either my volunteer time, or Zope Corp indirectly donating the value of my salary) -- we don't get a penny out of this. > (1) Judging from the traffic on c.l.py, Python seems to have reached > the point in a language's life cycle where it is attracting many novice > users. My inbox is a better judge of that . It sure is. > My guess is that most of those will be using the win32 distribution. Seems likely. > Many of them will no doubt refuse to RTFM, but for those who will, > providing them with the docs in .chm format (and actively promoting it) > might cut down on the "support cost" of the traffic on c.l.py. Perhaps ActiveState could address that from experience. > (2) In two application development shops with which I am engaged, > the main application runs on a large Unix box, but the developers' and > DBAs' workstations are Windows (2000) PCs -- is this atypical? Don't know. > I am evangelising Python to them -- while certainly not critical, > better doc formats, positive attitude to win32 distro, etc etc > wouldn't hurt. I'm committed to producing a working Win32 installer for the core Python distribution. PLabs can't afford to commit to more than that unless someone steps up to fund it, or volunteers to do new work. I have a sterling attitude toward peace in the Middle East too . From sjmachin@lexicon.net Sat Apr 6 01:43:14 2002 From: sjmachin@lexicon.net (John Machin) Date: Sat, 06 Apr 2002 11:43:14 +1000 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) In-Reply-To: <200204060033.g360Xit22223@pcp742651pcs.reston01.va.comcast.net> Message-ID: 06/04/2002 10:33:44 AM, Guido van Rossum wrote: > >> And some afterthoughts on Tim's tagline, which in my ignorance I >> interpreted as indicative of a somewhat reactive attitude to the >> win32 distro [I'd be utterly delighted to be corrected if I'm >> wrong]: > >You're wrong. I'm utterly delighted to be corrected. Thanks. >> I am evangelising Python to them -- while certainly not critical, >> better doc formats, positive attitude to win32 distro, etc etc >> wouldn't hurt. > >Stop complaining and do something about it. Wasn't a complaint -- I was enquiring why the .chm format wasn't included as standard; you and other respondents have kindly clued me in: (a) PITA to produce (b) mixed opinions as to whether it's better than HTML. I would do something if I thought I had any expertise in the area, but I don't. > If you think Tim's got an >negative attitude about win32 distro, I think you need a reality >adjustment. Would that be the same timbot to which I sent win32 versions of patch and diff way back when it was still running under win95 so that it could repair/enhance itself? If so, I don't think it has such an attitude :-) The word I used was "reactive" (not "negative") --- I got the impression from Tim's tagline that the general Pythonlabs attitude (not Tim's) was "we'll fix bugs in the win32 distro but we're not actively promoting it". Again: I'm utterly delighted to be corrected. Thanks. Best regards, John From guido@python.org Sat Apr 6 01:49:55 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 20:49:55 -0500 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) In-Reply-To: Your message of "Fri, 05 Apr 2002 20:15:33 EST." <02f701c1dd08$cecea2d0$6401a8c0@boostconsulting.com> References: <200204060033.g360Xit22223@pcp742651pcs.reston01.va.comcast.net> <02f701c1dd08$cecea2d0$6401a8c0@boostconsulting.com> Message-ID: <200204060149.g361ntZ22477@pcp742651pcs.reston01.va.comcast.net> > > > My guess is that most of those will be using the win32 > > > distribution. Many of them will no doubt refuse to RTFM, but for > > > those who will, providing them with the docs in .chm format (and > > > actively promoting it) might cut down on the "support cost" of the > > > traffic on c.l.py. > > > > I strongly doubt that the *format* we use to provide the docs makes > > much of a difference. > > *Strongly* disagree. As someone who doesn't know where everything is > using docs where many things are in the wrong place, having built-in > fast searching is absolutely essential for my work. I can just imagine > how it is for a novice... as someone who's got the whole world in his > brain, Guido, I think you should consider disqualifying yourself from > trying to intuit what people need in the documentation department. Fair enough, I hadn't thought of the search feature. When I use it on MS provided help topics, it's often very frustrating, so maybe I didn't think favorably of it. But I agree being able to search the docs is important. --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@pythoncraft.com Sat Apr 6 01:55:25 2002 From: aahz@pythoncraft.com (Aahz) Date: Fri, 5 Apr 2002 20:55:25 -0500 Subject: [Python-Dev] Namespaces In-Reply-To: <15534.18931.480692.612031@slothrop.zope.com> References: <002901c1d7f2$8b04f560$2eb53bd0@othello> <20020401025823.GA29354@panix.com> <20020401185657.GA6094@panix.com> <15528.51794.642978.22967@slothrop.zope.com> <20020401212232.GA14508@panix.com> <20020406004705.GA751@panix.com> <15534.18931.480692.612031@slothrop.zope.com> Message-ID: <20020406015524.GA9543@panix.com> On Fri, Apr 05, 2002, Jeremy Hylton wrote: > > I don't think that setting or getting an attribute is a name binding > operation. Maybe yes and maybe no -- we'll argue about that later. But do you agree that it's still a binding operation of some sort? If so, what's the generic term for all possible things that can be bound *from*? (They bind to objects, of course.) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From guido@python.org Sat Apr 6 02:02:01 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 21:02:01 -0500 Subject: [Python-Dev] Deprecating string exceptions In-Reply-To: Your message of "Fri, 05 Apr 2002 20:03:53 EST." <15534.18809.964868.250044@slothrop.zope.com> References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <15534.14484.134938.791320@slothrop.zope.com> <200204060018.g360IKI22119@pcp742651pcs.reston01.va.comcast.net> <15534.18809.964868.250044@slothrop.zope.com> Message-ID: <200204060202.g36221e22571@pcp742651pcs.reston01.va.comcast.net> > There's no urgency at all but a bug report isn't the right place for > the discussin, nor is our group's private email. I didn't think > that raising the question in the appropriate forum constituted > emergency action :-). One of your messages sounded like you were impatiently waiting to commit your change, and I wondered why that was. > GvR> I don't think the patch is ready yet, see my comment on SF. If > GvR> we decide that anything needs to be done in this area I want to > GvR> look at it first, so please don't check anything in yet. > > Did I miss the consensus on inheriting from Exception? I tried to > review the earlier thread, but the early stuff didn't have much on > Exception and the later stuff seemed to be about documentation. So it > must be buried in the middle if it's there :-). > > At any rate, I don't understand why it is helpful for classes to > inherit from exception. There are different views on this, but fact is that we've documented this as a recommendation since class exceptions were first introduced, and I've certainly considered starting to warn about exceptions raised or tested for that are *not* subclasses of Exception. There was some discussion that concluded that it was a useful feature (e.g. because the Exception class defines a standard signature and some standard attributes), and it was even suggested that in the future the traceback could be stored on the exception instance rather than in a separate variable. This would pave the way to deprecating sys.exc_info(). I kind of like that idea, so I am all for striving to enforce this in the future. > I am also curious -- this is off-topic for exceptions -- what the > right way to check for a new-style class is? And what other things > define Py_TPFLAGS_HEAPTYPE? There is no way to distinguish a new-style class, because all type objects can be considered new-style classes. The only thing you could do is checking for "not classic-class". The HEAPTYPE flag is set only for new-style classes that are created dynamically; but eventually I expect that the built-in exceptions will be statically declared new-style classes. --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Sat Apr 6 02:31:35 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 5 Apr 2002 21:31:35 -0500 Subject: [Python-Dev] Namespaces References: <002901c1d7f2$8b04f560$2eb53bd0@othello> <20020401025823.GA29354@panix.com> <20020401185657.GA6094@panix.com> <15528.51794.642978.22967@slothrop.zope.com> <20020401212232.GA14508@panix.com> <20020406004705.GA751@panix.com> <15534.18931.480692.612031@slothrop.zope.com> <20020406015524.GA9543@panix.com> Message-ID: <15534.24071.152009.959329@anthem.wooz.org> >>>>> "JH" == Jeremy Hylton writes: JH> I don't think that setting or getting an attribute is a name JH> binding operation. I'm curious why setting an attribute isn't a name binding operation? Are you binding the object to the attribute name in the object's attribute namespace? Since that maps to a setting in an __dict__ (usually), what /would/ you call it? >>>>> "A" == Aahz writes: A> Maybe yes and maybe no -- we'll argue about that later. But do A> you agree that it's still a binding operation of some sort? Certainly a setter is, although we could argue about the getter (ignoring getters with side-effects). A> If so, what's the generic term for all possible things that can A> be bound *from*? (They bind to objects, of course.) I'm sure I don't follow. :) I'd say that a binding operation is that which binds an object to a name in a particular namespace. But I'd want to prefix the term "namespace" with an adjective to be precise. An object's attribute namespace, a module's namespace, etc. -Barry From barry@zope.com Sat Apr 6 02:35:18 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 5 Apr 2002 21:35:18 -0500 Subject: [Python-Dev] Deprecating string exceptions References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <15534.14484.134938.791320@slothrop.zope.com> <200204060018.g360IKI22119@pcp742651pcs.reston01.va.comcast.net> <15534.18809.964868.250044@slothrop.zope.com> <200204060202.g36221e22571@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15534.24294.725727.593378@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> There was some discussion that concluded that it was a useful GvR> feature (e.g. because the Exception class defines a standard GvR> signature and some standard attributes) But most of the (early) built-in exception interfaces were chosen for backwards compatibility with the old string-based standard exceptions, so I'm not sure how useful they are as a specification. E.g. Exception.__init__() essentially takes a *args and assigns the tuple to exc.args, and that's about it. Maybe in a perfect world that's still a useful signature, I don't know. -Barry From fdrake@acm.org Sat Apr 6 02:37:15 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 5 Apr 2002 21:37:15 -0500 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) In-Reply-To: <200204060033.g360Xit22223@pcp742651pcs.reston01.va.comcast.net> References: <200204060033.g360Xit22223@pcp742651pcs.reston01.va.comcast.net> <02f701c1dd08$cecea2d0$6401a8c0@boostconsulting.com> Message-ID: <15534.24411.659593.420529@grendel.zope.com> Guido van Rossum writes: > Speak for yourself. I prefer the HTML that I can view in a browser of > my choice any time over MS's clumsy help browser. HTML Help != WinHelp. WinHelp is the older and incredibly painful thing I suspect you're referring to. HTML Help is actually pretty nice. > I strongly doubt that the *format* we use to provide the docs makes > much of a difference. The HTML Help browser is substantially better than browsing a bunch of HTML files, primarily because of the index and full text search. > > (2) In two application development shops with which I am engaged, > > the main application runs on a large Unix box, but the developers' > > and DBAs' workstations are Windows (2000) PCs -- is this atypical? > > I've never seen it, but I suppose I'm atypical. :-) David Abrahams writes: > *Strongly* disagree. As someone who doesn't know where everything is > using docs where many things are in the wrong place, having built-in > fast searching is absolutely essential for my work. I can just imagine Yep. The regular revival of threads about the need for better ways to access the documentation certainly indicates we need to improve the situation. The only (substantial) reason I've not been building the HTML Help version is that I've not had the time to figure out how to do it. I don't think it's all that difficult, but I need to spend a little time with it and make sure I understand the tools. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From fdrake@acm.org Sat Apr 6 02:44:35 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 5 Apr 2002 21:44:35 -0500 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) In-Reply-To: <200204060033.g360Xit22223@pcp742651pcs.reston01.va.comcast.net> References: <200204060033.g360Xit22223@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15534.24851.671767.579978@grendel.zope.com> Guido van Rossum writes: > > (2) In two application development shops with which I am engaged, > > the main application runs on a large Unix box, but the developers' > > and DBAs' workstations are Windows (2000) PCs -- is this atypical? > > I've never seen it, but I suppose I'm atypical. :-) I've seen it, and I suspect it's pretty common. In the specific situation I worked in, we were developing a 3-tier vertical-market application. We had a server that provided database access with domain logic & security, and a "fat" GUI client for Windows. (For people who don't know what a 3-tier system is, the 3rd tier is the data storage layer -- Oracle 7 in our shop.) Our server ran on a variety of platforms: NT, several Unix flavors, and VMS. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From goodger@users.sourceforge.net Sat Apr 6 03:10:57 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Fri, 05 Apr 2002 22:10:57 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format In-Reply-To: <200204031934.g33JYfN22909@odiug.zope.com> Message-ID: Guido van Rossum wrote: > But if you ask me "should we use this for the standard library" I > think I'll have to say no. If the question is "Should we convert all library documentation and stuff it into docstrings in the source?", I would agree with you wholeheartedly. There are some people who would like it that way, but I'm not one of them. People seem to have gotten the impression that I'm advocating taking the library docs out of LaTeX and stuffing them into module docstrings, for later extraction & processing. That is *not* the case! I don't know *how* anyone got that idea! The docs are safe from me. ;-) > Given this status quo, docstrings in the Python standard library > should not try to duplicate the library reference documentation; > instead, they should be no more than concise hints. I agree completely. > For such docstrings, a markup language, even reStructuredText, is > inappropriate. > > IOW, reStructuredText is not even an option for new standard library > modules. What about existing docstrings? There is plenty of informal markup in there already. For the standard library I would suggest that, once the tools are up to it (e.g., once there's reStructuredText support in pydoc), *existing* docstrings could be *minimally* converted to formalize the implicit markup that's *already there*. For example, here's the module docstring for the string.py module: """A collection of string operations (most are no longer used in Python 1.6). Warning: most of the code you see here isn't normally used nowadays. With Python 1.6, many of these functions are implemented as methods on the standard string object. They used to be implemented by a built-in module called strop, but strop is now obsolete itself. Public module variables: whitespace -- a string containing all characters considered whitespace lowercase -- a string containing all characters considered lowercase l uppercase -- a string containing all characters considered uppercase l letters -- a string containing all characters considered letters digits -- a string containing all characters considered decimal digits hexdigits -- a string containing all characters considered hexadecimal octdigits -- a string containing all characters considered octal digit punctuation -- a string containing all characters considered punctuati printable -- a string containing all characters considered printable """ (I wrapped the first two paragraphs and truncated the list so email wouldn't wreck it.) As it stands, this is almost valid reStructuredText (strictly speaking it is already valid, but the list would get wrapped and wouldn't be very useful). The list of variables needs a bit of work; it could be turned into a bullet list or a definition list. The variable identifiers themselves could be marked up as "interpreted text" (e.g. rendered in a different face, with links to each identifier's docstring if it exists). The warning could be left as-is, or spruced up. Here is the fully converted docstring: """A collection of string operations (most are no longer used in Python 1.6). .. Warning:: most of the code you see here isn't normally used nowadays. With Python 1.6, many of these functions are implemented as methods on the standard string object. They used to be implemented by a built-in module called strop, but strop is now obsolete itself. Public module variables: `whitespace` a string containing all characters considered whitespace `lowercase` a string containing all characters considered lowercase letters `uppercase` a string containing all characters considered uppercase letters `letters` a string containing all characters considered letters `digits` a string containing all characters considered decimal digits `hexdigits` a string containing all characters considered hexadecimal digits `octdigits` a string containing all characters considered octal digits `punctuation` a string containing all characters considered punctuation `printable` a string containing all characters considered printable """ The conversion is minimal (it could be even less), it's still perfectly readable, and the difference in the converted output is significant. Please take a look at the converted output (1 or 2) and compare to the output for vanilla pydoc (3). 1. http://structuredtext.sf.net/spec/string.html 2. http://structuredtext.sf.net/spec/string2.html (bullet list instead of definition list) 3. the first section of http://web.pydoc.org/2.2/string.html (Note that the HTML uses a CSS1 stylesheet, so a recent browser is required. A writer for HTML for older browsers is on the to-do list.) In any case, nothing needs to be done any time soon. What do you think? > I agree with Jeremy that the PEP needs to be clear and explicit > about this. Will do. -- David Goodger goodger@users.sourceforge.net Open-source projects: - Python Docstring Processing System: http://docstring.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net - The Go Tools Project: http://gotools.sourceforge.net From guido@python.org Sat Apr 6 03:16:12 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 22:16:12 -0500 Subject: [Python-Dev] Deprecating string exceptions In-Reply-To: Your message of "Fri, 05 Apr 2002 21:35:18 EST." <15534.24294.725727.593378@anthem.wooz.org> References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <15534.14484.134938.791320@slothrop.zope.com> <200204060018.g360IKI22119@pcp742651pcs.reston01.va.comcast.net> <15534.18809.964868.250044@slothrop.zope.com> <200204060202.g36221e22571@pcp742651pcs.reston01.va.comcast.net> <15534.24294.725727.593378@anthem.wooz.org> Message-ID: <200204060316.g363GCG22664@pcp742651pcs.reston01.va.comcast.net> > GvR> There was some discussion that concluded that it was a useful > GvR> feature (e.g. because the Exception class defines a standard > GvR> signature and some standard attributes) > > But most of the (early) built-in exception interfaces were chosen for > backwards compatibility with the old string-based standard exceptions, > so I'm not sure how useful they are as a specification. > E.g. Exception.__init__() essentially takes a *args and assigns the > tuple to exc.args, and that's about it. Maybe in a perfect world > that's still a useful signature, I don't know. Well, it could certainly grow more standard stuff (like a traceback pointer) and that would be more useful if there was a common base class to inherit it from. --Guido van Rossum (home page: http://www.python.org/~guido/) From goodger@users.sourceforge.net Sat Apr 6 03:25:24 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Fri, 05 Apr 2002 22:25:24 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format In-Reply-To: <15531.20700.225267.683023@slothrop.zope.com> Message-ID: Thanks Jeremy, for raising good points. Jeremy Hylton wrote in his first message: > I have always been puzzled by why Python has taken so long to come > up with some simple conventions for structuring docstrings. I think there are many factors. People have different requirements, and often insist that their minimal set is the best set (thus reStructuredText is more flexible). We've seen what else is out there, and we aren't satisfied. Python has shown us how readable code can be, and we strive for readable inline docs as well. Nobody has been as committed, driven, and nuts as me before. > The one time I looked at JavaDoc, it struck me as quite simple Simple, yes, but ugly as sin. I wouldn't want JavaDoc in *my* modules. Many others concur. > It also appeared that JavaDoc had a limited feature set, which also > seemed like a strength: Let's not write fancy, formatted reports in > docstrings. This is a common argument, but I think a flawed one. It assumes that rich syntax inevitably begets over-complex docs, and at the same time assumes that no docs ever need to use sophisticated features. Fine, don't write fancy docstrings. But what if you need a single little table? Or a definition list (like in string.py; see my message to Guido)? Eventually you'll need something more sophisticated, but the markup is limited. You're stuck, you get frustrated, throw the tool out and revert to plaintext with ad-hoc conventions. The Python equivalent would be to limit the feature set by eliminating, say, floats. Ints ought to be enough for anyone! Those floats just cause problems... > It looks like I can mostly read it, but it seems hard to learn how > to write. I think that's because I only gave the spec, without the primer. Now that we have a primer, what do you think? > It seems burdensome to require module authors to learn Python and > reStructuredText just to contribute to the std library. I'm not advocating that, never have. I'll make it clear in the PEP. Jeremy wrote in his second message: > If the primary goal is to keep the markup simple... ... > A good design for a docstring format makes some hard decisions about > what actually is essential and what is bloat. The primary goal isn't just "simple", it's "readable, simple, rich, easy, and extensible plaintext" (among other things; see http://structuredtext.sf.net/spec/introduction.html#goals). These goals obviously conflict, and the design of the markup has been an exercise in compromise. A successful one, I think. I tried to make simple and common constructs simple. Rarer constructs are more verbose, but since they're rare, it's not such a big deal. Jeremy wrote in his third message: > I wouldn't object to seeing this PEP approved as an informational > PEP that described reST as an optional format for docstrings. That's all I ever intended; sorry if it seemed otherwise. > (I'm assuming that there is consensus in the doc-sig that reST is > the right solution.) More than ever before, I think. 100% consensus is rarely attained in any forum (this one included). -- David Goodger goodger@users.sourceforge.net Open-source projects: - Python Docstring Processing System: http://docstring.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net - The Go Tools Project: http://gotools.sourceforge.net From guido@python.org Sat Apr 6 03:34:47 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 22:34:47 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format In-Reply-To: Your message of "Fri, 05 Apr 2002 22:10:57 EST." References: Message-ID: <200204060334.g363Ym922772@pcp742651pcs.reston01.va.comcast.net> [I'm hoping David Goodger is reading this on python-dev. Replying to @users.sf.net doesn't work from my home box. It's my fault but I have no idea how to fix it.] > > IOW, reStructuredText is not even an option for new standard library > > modules. > > What about existing docstrings? There is plenty of informal markup in > there already. Yeah, but they're not using any particular formal markup. You'd still have to do a massive, *massive* cleanup if you wanted reST to apply. And it's not clear what the advantage would be -- stdlib docstrings are mostly intended as hints, summarizing the info from the library reference, and as such are intended for presentation *without* additional processing. Minimal though reST is, it's still markup. > For the standard library I would suggest that, once the tools are up > to it (e.g., once there's reStructuredText support in pydoc), > *existing* docstrings could be *minimally* converted to formalize the > implicit markup that's *already there*. For example, here's the > module docstring for the string.py module: > > """A collection of string operations (most are no longer used in > Python 1.6). > > Warning: most of the code you see here isn't normally used nowadays. > With Python 1.6, many of these functions are implemented as methods on > the standard string object. They used to be implemented by a built-in > module called strop, but strop is now obsolete itself. > > Public module variables: > > whitespace -- a string containing all characters considered whitespace > lowercase -- a string containing all characters considered lowercase l > uppercase -- a string containing all characters considered uppercase l > letters -- a string containing all characters considered letters > digits -- a string containing all characters considered decimal digits > hexdigits -- a string containing all characters considered hexadecimal > octdigits -- a string containing all characters considered octal digit > punctuation -- a string containing all characters considered punctuati > printable -- a string containing all characters considered printable > > """ > > (I wrapped the first two paragraphs and truncated the list so email > wouldn't wreck it.) > > As it stands, this is almost valid reStructuredText (strictly speaking > it is already valid, but the list would get wrapped and wouldn't be > very useful). The list of variables needs a bit of work; it could be > turned into a bullet list or a definition list. The variable > identifiers themselves could be marked up as "interpreted text" > (e.g. rendered in a different face, with links to each identifier's > docstring if it exists). The warning could be left as-is, or spruced > up. Here is the fully converted docstring: > > """A collection of string operations (most are no longer used in > Python 1.6). > > .. Warning:: most of the code you see here isn't normally used > nowadays. With Python 1.6, many of these functions are implemented > as methods on the standard string object. They used to be > implemented by a built-in module called strop, but strop is now > obsolete itself. > > Public module variables: > > `whitespace` > a string containing all characters considered whitespace > `lowercase` > a string containing all characters considered lowercase letters > `uppercase` > a string containing all characters considered uppercase letters > `letters` > a string containing all characters considered letters > `digits` > a string containing all characters considered decimal digits > `hexdigits` > a string containing all characters considered hexadecimal digits > `octdigits` > a string containing all characters considered octal digits > `punctuation` > a string containing all characters considered punctuation > `printable` > a string containing all characters considered printable > > """ > > The conversion is minimal (it could be even less), it's still > perfectly readable, and the difference in the converted output is > significant. Please take a look at the converted output (1 or 2) and > compare to the output for vanilla pydoc (3). > > 1. http://structuredtext.sf.net/spec/string.html > 2. http://structuredtext.sf.net/spec/string2.html (bullet list instead > of definition list) > 3. the first section of http://web.pydoc.org/2.2/string.html > > (Note that the HTML uses a CSS1 stylesheet, so a recent browser is > required. A writer for HTML for older browsers is on the to-do list.) > > In any case, nothing needs to be done any time soon. What do you > think? Alas, I find both the input and the output of the reST-ized version worse than the original. The original takes up much less vertical space, which (given the goal of being a relatively terse hint rather than formal reference docs) counts for more than bullets, boxes and color. The situation would be different if the goal was to replace the reference docs, but since it isn't, I think the informal markup is just fine. --Guido van Rossum (home page: http://www.python.org/~guido/) From goodger@users.sourceforge.net Sat Apr 6 04:15:45 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Fri, 05 Apr 2002 23:15:45 -0500 Subject: [Python-Dev] Re: PEP 287: reStructuredText Standard Docstring Format In-Reply-To: <005501c1dc95$b17a3aa0$6d94fea9@newmexico> Message-ID: Thanks for the many replies, Samuele. I've put all my responses here. [PEP 287] > 1. To establish a standard docstring format by attaining > "accepted" status (Python community consensus; BDFL > pronouncement). ... [Samuele Pedroni] > Really? establish in what sense? PEP acceptance; community awareness; eventually culminating in tools in the stdlib. This is just one step. [PEP 287] > Once a standard exists, people will start to use > it, and momentum will inevitably gather. [Samuele Pedroni] > This is rather naive. Why do Perl programmers write inline docs in POD? Because it's the Perl standard, not because it's a great markup language. Same for Java and JavaDoc. I'm not satisfied by those examples, barely adequate IMHO. Instead, I'm working on something *better* for Python. Many people with experience in JavaDoc say "What's wrong with JavaDoc?" I tend to discount those opinions, for the same reason I discount C programmers decrying Python's lack of variable declarations or braces: they've been tainted by experience, and aren't entirely objective. (I don't claim to be, either, but I try to keep an open mind.) Not everybody agrees, true. Not everbody agrees on *anything*, even seemingly innocuous additions to Python like booleans. PEP 287 doesn't introduce *any* changes to the language, just an optional markup for docstrings. People are already starting to use reStructuredText. There are already spin-off projects underway; PythonPoint integration is one. I've received several unsolicited email testimonials in support. I've seen others on the web; do a Google search for "restructuredtext" and you'll find some. > it seems that the PEP is the result of a motivated self-selected > group with difficulty to reach consensus and so neglecting > minimality... There is some truth to that. There is a *lot* of relevant history on the Doc-SIG. ReStructuredText and PEP 287 are influenced by that history. Minimality is an oft-lauded goal, but minimal tools and markup languages tend to be limited. > From the PEP: > > [[Are these in fact the goals of the Doc-SIG members? Anything > to add?]] > > Throughout the existence of the Doc-SIG, consensus on a single > standard docstring format has never been reached. > > so is this just your proposal? how should we parse that? The question [[in brackets]] was from a Doc-SIG post directed at Doc-SIG members, before I posted to python-dev and comp.lang.python. You must have read the PEP from there, from CVS, or from python.org/peps/pep-0287.(txt|html). That question was answered by Doc-SIG members and was removed from the version I posted to python-dev and comp.lang.python. Sorry you had to see that. > The PEP focus seems out of focus ... > I don't see a clear picture of the input -> output relation in all > the PEPs, I see now that since it is part of a greater whole, the PEP needs to locate itself with respect to the other parts; a "big picture" overview, or a map with a big "You Are Here". Then I think the focus will become clear. I will add such a map. > it seems that the PEP goes well beyond the kind of structure that > the average programmer need Who is "the average programmer"? I've learned from experience with markup that an artificially limited feature set will limit adoption. If that weren't the case, everyone would be using HappyDoc (the tool is good, but the markup -- StructuredText -- is limited). HappyDoc + reStructuredText could be a killer combination. Aim for mediocrity and we may get there, but we'll never reach excellence. I prefer to aim for excellence. > what happens if I put 3 level of titles in the doc of a method? You may get crappy docs; that's the author's problem. But sometimes, 3 levels of titles may be necessary and appropriate! If the markup doesn't let you do something when you need it, you'll curse it and stop using it. > oops, ye the abstract says that the PEP is not concerned with this True, it isn't. That's intentional. The Python docs don't tell you what kind of programs you can or cannot write, either. > the goal as stated are rich docstrings (for what?) For inline documentation of Python modules and packages. I'll clarify. > I'm puzzled, the PEP does not even refer or cite PEP 257, which OTOH > seem also not to address completly the issue of doc-extraction tools > interaction. > > Honestly, PEP 256,257,258, PEP 287 where are we going? As I've said, it's a big issue. My approach has been to split it up and deal with the pieces individually. No other approach has worked thus far... Would it be better if I combined everything into one mega-PEP? Roughly novel-length, I imagine! > Maybe goal 1 should be honestly rephrased (Not that it was ever dishonest, but) How's this? 1. To establish reStructuredText as a standard structured plaintext format for docstrings (inline documentation of Python modules and packages), PEPs, README-type files and other standalone documents. "Accepted" status will be sought through Python community consensus and eventual BDFL pronouncement. Please note that reStructuredText is being proposed as *a* standard, not *the only* standard. Its use will be entirely optional. Those who don't want to use it need not. > you got at least the politics wrong IMHO You got that right! Nobody ever accused me of being a politician... [PEP 287] > ... They may be used to implement docstring semantics, such as > identifying parameters, exceptions raised, etc.; such usage is > beyond the scope of this PEP. [Samuele] > how many eons would pass before we see a PEP about > those *relevant* issues? I don't know the answer to that one. That's a policy issue, and I won't be touching it until the technical issues (writing the code) is much further along, if ever. I think docstring semantics may be the thorniest issue of all; people can accept a *syntax*, which is low-level, but telling them how to write docs from a high level? Frederic Giacometti's "iPhrase Python documentation conventions" is an attachment to his Doc-SIG post of 2001-05-30 (http://mail.python.org/pipermail/doc-sig/2001-May/001840.html). Interesting, but didn't go anywhere. The approach I have taken is to break up the issues and try deal with each of them in isolation, or as close as I can get. IMO one cause of the failures of other tools is that they imposed a fixed set of (input syntax, semantics [docstring conventions and policies], processing mechanisms, output formats, styles) on the users. None of them have gotten this set just right; I don't think it's possible to get it right, at least not initially. So I've broken them up, and the Docutils is being designed modularly so that any of these components can be swapped out easily. > Presumably JavaDoc is a good start point for this . Yes, it's a good start in some ways. > More or less JavaDoc tags would map to fields ReStructuredText field lists were partially modelled on JavaDoc tags. The JavaDoc tag syntax was also considered but rejected (see the alternatives doc for details). [from PEP 287] > The lack of a standard syntax for docstrings has hampered the > development of standard tools ... [Samuele Pedroni] > Honestly, is that true? PEP 256 lists several tools, of which only two are current: pydoc and HappyDoc. Pydoc is in the standard library, but it has two deficiencies: it doesn't deal with markup *at all* (just treats docstrings as literal blocks or ); and it imports modules to document, thereby losing information and potentially incurring a security risk. HappyDoc works well, but uses the older StructuredText markup. From HappyDoc's README: *How does an author write documentation so that it will be marked up and look fancy?* This is a perennial question for Python users, and seems to have introduced a roadblock into the development of more robust and useful documentation tools. HappyDoc stands firmly on the fence and does not attempt to resolve the issue. ReStructuredText is a more robust and useful markup, and Docutils intends to be a more robust and useful toolset. I hope that HappyDoc (possibly with contributions from other tools) can inspire, influence, subsume, or itself *become* Docutils. > how does the PEP relate to producing from inline docstrings at least > a first cut in a subset of the py-tex-markup format? It doesn't. It deals with input markup only. At some point Docutils should have a py-tex-markup writer component. The Docutils PEP (258) talks about output formats. > I *honestly* wonder whether to the Python party really corresponds > more-or-less just one documentation format party, a silent majority > perhaps. About as much as the Perl or Java parties, I expect. The only difference is they already have a doc format, and we don't. When there's one obvious choice, well, that's obviously what's chosen. > And whether Docutils (formerly DPS) will be up to this ambitious > design claim in PEP 258 (revision 1.2): > > Intermediate Data Structure > =========================== > ... > The DTD defines a rich set of elements, suitable for any input > syntax or output format The DTD is quite generic, but it has been influenced by the one and only markup available. Given another markup, it would be even more generic. I try not to generalize prematurely. > PS: the funny thing is that I do not dislike structured text formats > in general, but auto-documenting is reducible to that ? No, of course not. It's just one piece of the puzzle. -- David Goodger goodger@users.sourceforge.net Open-source projects: - Python Docstring Processing System: http://docstring.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net - The Go Tools Project: http://gotools.sourceforge.net From goodger@users.sourceforge.net Sat Apr 6 04:16:25 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Fri, 05 Apr 2002 23:16:25 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format In-Reply-To: <04d301c1dc37$e8a194a0$6d94fea9@newmexico> Message-ID: [Samuele Pedroni, re docstring semantics] >>> how many eons would pass before we see a PEP about >>> those *relevant* issues? Presumably JavaDoc is a good >>> start point for this . [Fredrik Lundh] > > do you need a PEP, or is it okay if I just post some code? [Samuele Pedroni] > Code please, see code is less controversial than PEPs... Go ahead, Fredrik, post some code. If we're lucky, it won't be piled on to the trash heap of past experiments. If you're serious and not just being snide, I look forward to the results. -- David Goodger goodger@users.sourceforge.net Open-source projects: - Python Docstring Processing System: http://docstring.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net - The Go Tools Project: http://gotools.sourceforge.net From David Abrahams" <200204060334.g363Ym922772@pcp742651pcs.reston01.va.comcast.net> Message-ID: <03ca01c1dd23$44219550$6401a8c0@boostconsulting.com> From: "Guido van Rossum" > Alas, I find both the input and the output of the reST-ized version > worse than the original. The original takes up much less vertical > space, which (given the goal of being a relatively terse hint rather > than formal reference docs) counts for more than bullets, boxes and > color. The situation would be different if the goal was to replace > the reference docs, but since it isn't, I think the informal markup is > just fine. I have to agree with Guido, for the same reasons. I wonder if it's the input markup that's "wrong", or if it just needs a better backend for generating HTML from the markup. Incidentally, I'm really excited about reST. I've been looking for a tolerable markup for C++ comments, and reST looks like it might fit the bill. Other musings: A reST-based Wiki would give people an easy way to see how they like the format. Is it just me, or are docstrings less-convenient than comments? Any thought given to reST-processing a module's comments? -Dave From pedroni@inf.ethz.ch Sat Apr 6 04:36:04 2002 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Sat, 6 Apr 2002 06:36:04 +0200 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format References: <200204060334.g363Ym922772@pcp742651pcs.reston01.va.comcast.net> <03ca01c1dd23$44219550$6401a8c0@boostconsulting.com> Message-ID: <004501c1dd24$8fe07e60$6d94fea9@newmexico> > > Is it just me, or are docstrings less-convenient than comments? Any > thought given to reST-processing a module's comments? > Maybe it's just me but I would be more happy to optionally be able to write reST inside comments instead of docstrings and have it extracted from there. That means the comment would take the place of the secondary docstring. regards. From aahz@pythoncraft.com Sat Apr 6 04:39:48 2002 From: aahz@pythoncraft.com (Aahz) Date: Fri, 5 Apr 2002 23:39:48 -0500 Subject: [Python-Dev] Namespaces In-Reply-To: <15534.24071.152009.959329@anthem.wooz.org> References: <20020401025823.GA29354@panix.com> <20020401185657.GA6094@panix.com> <15528.51794.642978.22967@slothrop.zope.com> <20020401212232.GA14508@panix.com> <20020406004705.GA751@panix.com> <15534.18931.480692.612031@slothrop.zope.com> <20020406015524.GA9543@panix.com> <15534.24071.152009.959329@anthem.wooz.org> Message-ID: <20020406043948.GA1681@panix.com> On Fri, Apr 05, 2002, Barry A. Warsaw wrote: > > A> If so, what's the generic term for all possible things that can > A> be bound *from*? (They bind to objects, of course.) > > I'd say that a binding operation is that which binds an object to a > name in a particular namespace. But I'd want to prefix the term > "namespace" with an adjective to be precise. An object's attribute > namespace, a module's namespace, etc. So we have * primary names (gotten from the lang ref), which are names in the local/global/builtin namespace * object names (and a module is an object ;-) AKA attributes * computed bindings (getitem/setitem) What do we call all three of these collectively? I'd say that I agree with MvL that computed bindings are not in fact names, and as I said in my other post, using "binding" as the collective noun looks ugly when we're also using it as a verb. The key is to come up with some good way of saying When rebinding a ... -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From goodger@users.sourceforge.net Sat Apr 6 04:44:44 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Fri, 05 Apr 2002 23:44:44 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format In-Reply-To: <200204031934.g33JYfN22909@odiug.zope.com> Message-ID: Guido van Rossum wrote: > [I'm hoping David Goodger is reading this on python-dev. Yes, I do. I read it off the web. I know I can now subscribe without going through the initiation ceremony and learning the secret handshake, but old habits die hard. > > What about existing docstrings? There is plenty of informal markup in > > there already. > Yeah, but they're not using any particular formal markup. You'd still > have to do a massive, *massive* cleanup if you wanted reST to apply. > And it's not clear what the advantage would be ... > The situation would be different if the goal was to replace > the reference docs, but since it isn't, I think the informal markup is > just fine. Fair enough. This is a long-term project. Plenty of time to change your mind later, when the advantages become clear. :-) -- David Goodger goodger@users.sourceforge.net Open-source projects: - Python Docstring Processing System: http://docstring.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net - The Go Tools Project: http://gotools.sourceforge.net From DavidA@ActiveState.com Sat Apr 6 04:51:52 2002 From: DavidA@ActiveState.com (David Ascher) Date: Fri, 05 Apr 2002 20:51:52 -0800 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar forMozilla/Netscape) References: Message-ID: <3CAE7EE8.1C3445AA@ActiveState.com> Tim Peters wrote: > Sure. Packing over 1,000 HTML files into the installer bloats it too Indeed. ActivePython's installer is significantly bigger than the Pythonlabs installer, because we have a lot more docs in there (not just the Pythonlabs docs -- the PyWin32 docs, others.). Our installer seems to do a good job of compression, but it's still a lot of text. > Possibly, but the idea that a new software process involving many megabytes > of data won't create new time sinks isn't credible to me. Fred takes pride > in the appearance of his docs, so won't want to let small points slide > either. There is indeed flexibility in having the two-phase release cycle, which sees Fred updating the docs several times per binary-release-cycle. That's nice, given that the documentation naturally lags implementation (and will as long as the product is managed by developers =). Also, figuring out how to create HTML files which look good on the website, in HtmlHelp and in regular browsers is a real pain for people like me who don't understand why browsers simply had to be so incompatible =). > > Many of them will no doubt refuse to RTFM, but for those who will, > > providing them with the docs in .chm format (and actively promoting it) > > might cut down on the "support cost" of the traffic on c.l.py. > > Perhaps ActiveState could address that from experience. We spend a lot of time and effort making all of our tools as self-sufficient as possible. We also do a lot of work so that the website (ASPN.ActiveState.com) provides as much help on both products and technologies as possible, since when that fails, we incur developer and support staff work, which doesn't scale and costs a lot. I'm not sure that that equation is particularly relevant for Pythonlabs, though -- most of the questions on c.l.py are answered by non-Pythonlabers, so the benefit of investing in a somewhat better distro isn't really apparent to the bottom line, while the development cost is. Pythonlabs doesn't have an obligation to answer every question sent to python-list -- that burden is carried quite well by the community, as it always has, and as it should be. It may sound weird to mention the bottom line w.r.t. how an open source project is run, but a big reason Python is where it is is because someone (in practice, several someones) have paid Guido et al. to do this work. So the bottom line matters -- and it is important for Python's long-term future that the Pythonlabers take the responsibility of thinking about where they spend their time as seriously as they do. This week, it may seem to Pythonlabers that the users are being cranky and whining (this is not directed at John's post, just at the general tenor of comp.lang.python in the last few days). I want to publically state that I think the Pythonlabs folks are doing an admirable job, above and beyond the call of duty. Python _is_ evolving faster than it did in the middle-early days (I expect the versions before 1.2 evolved very fast), and our community is growing as a result. The growth is a strong sign of health, and seems to indicate that the language evolution is overall in the positive, niche-expanding direction. I am convinced that some recent developments such as the better release management, the filling in of the standard library and documentation, the cleanup and maturing of the language are all factors which overall much outweigh the pains that some feel with respect to managing multi-version code bases or coping with language evolution. At the end of the day, Python is still just as fun to program in as it ever was. --David "cheerleader" Ascher From barry@zope.com Sat Apr 6 05:09:52 2002 From: barry@zope.com (Barry A. Warsaw) Date: Sat, 6 Apr 2002 00:09:52 -0500 Subject: [Python-Dev] Deprecating string exceptions References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <15534.14484.134938.791320@slothrop.zope.com> <200204060018.g360IKI22119@pcp742651pcs.reston01.va.comcast.net> <15534.18809.964868.250044@slothrop.zope.com> <200204060202.g36221e22571@pcp742651pcs.reston01.va.comcast.net> <15534.24294.725727.593378@anthem.wooz.org> <200204060316.g363GCG22664@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15534.33568.340660.611621@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> Well, it could certainly grow more standard stuff (like a GvR> traceback pointer) and that would be more useful if there was GvR> a common base class to inherit it from. Agreed. Of course you can always play the old NeXTSTEP game of having one base class for backwards compatibility and another for the New Way, but that's probably more complexity than you really want. -Barry From pedroni@inf.ethz.ch Sat Apr 6 05:15:51 2002 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Sat, 6 Apr 2002 07:15:51 +0200 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format References: <200204060334.g363Ym922772@pcp742651pcs.reston01.va.comcast.net> <03ca01c1dd23$44219550$6401a8c0@boostconsulting.com> <004501c1dd24$8fe07e60$6d94fea9@newmexico> Message-ID: <014801c1dd2a$1ece8360$6d94fea9@newmexico> From: Samuele Pedroni >From David Abrahams > > > > Is it just me, or are docstrings less-convenient than comments? Any > > thought given to reST-processing a module's comments? > > > > Maybe it's just me but I would be more happy to optionally > be able to write reST inside comments instead of docstrings > and have it extracted from there. That means > the comment would take the place of the secondary > docstring. > that means optionally e.g: # identifiable heavy reST # def f(... """plain text or light reST""" vs. def f(.. """plain text or light reST""" """heavy reST""" regards. From barry@zope.com Sat Apr 6 05:26:37 2002 From: barry@zope.com (Barry A. Warsaw) Date: Sat, 6 Apr 2002 00:26:37 -0500 Subject: [Python-Dev] Namespaces References: <20020401025823.GA29354@panix.com> <20020401185657.GA6094@panix.com> <15528.51794.642978.22967@slothrop.zope.com> <20020401212232.GA14508@panix.com> <20020406004705.GA751@panix.com> <15534.18931.480692.612031@slothrop.zope.com> <20020406015524.GA9543@panix.com> <15534.24071.152009.959329@anthem.wooz.org> <20020406043948.GA1681@panix.com> Message-ID: <15534.34573.142067.134677@anthem.wooz.org> >>>>> "A" == Aahz writes: A> * primary names (gotten from the lang ref), which are names in A> the local/global/builtin namespace A> * object names (and a module is an object ;-) AKA attributes I'd just call these "attributes" . Okay, maybe "attribute names". A> * computed bindings (getitem/setitem) I'd call these "computed attributes". I don't like "computed bindings" because that sounds like you're saying that you're computing both the target of the binding and the object that is being bound (or one or the other?). A> What do we call all three of these collectively? I'd say that A> I agree with MvL that computed bindings are not in fact names, "Computed attributes" aren't bindings, but that only matters if you can tell the difference. In a normal situation, if you see "obj.foo" it may not matter that "foo" only exists by virtue of a setattr/getattr/descriptor. I'd still claim that "foo" is an attribute name because it's part of the public interface for obj, albeit perhaps a virtual attribute name. A> and as I said in my other post, using "binding" as the A> collective noun looks ugly when we're also using it as a verb. A> The key is to come up with some good way of saying A> When rebinding a ... Names get bound, so names get rebound. At least, that's how I think about it. semantical-ly y'rs, -Barry From jeremy@zope.com Sat Apr 6 05:38:07 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Sat, 6 Apr 2002 00:38:07 -0500 Subject: [Python-Dev] Deprecating string exceptions In-Reply-To: <15534.33568.340660.611621@anthem.wooz.org> References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <15534.14484.134938.791320@slothrop.zope.com> <200204060018.g360IKI22119@pcp742651pcs.reston01.va.comcast.net> <15534.18809.964868.250044@slothrop.zope.com> <200204060202.g36221e22571@pcp742651pcs.reston01.va.comcast.net> <15534.24294.725727.593378@anthem.wooz.org> <200204060316.g363GCG22664@pcp742651pcs.reston01.va.comcast.net> <15534.33568.340660.611621@anthem.wooz.org> Message-ID: <15534.35263.155509.971861@slothrop.zope.com> >>>>> "BAW" == Barry A Warsaw writes: >>>>> "GvR" == Guido van Rossum writes: GvR> Well, it could certainly grow more standard stuff (like a GvR> traceback pointer) and that would be more useful if there was a GvR> common base class to inherit it from. BAW> Agreed. Of course you can always play the old NeXTSTEP game of BAW> having one base class for backwards compatibility and another BAW> for the New Way, but that's probably more complexity than you BAW> really want. I'm not sure what the base class(es) actually buys us. We could just as well say by fiat that the interpreter will set the traceback attribute of an exception instance to the traceback. Why do we need a special base class to accomplish that? This is Python. You don't need to do isinstance(). You just need to see if it has the right attributes. BTW, what is the signature for Exception.__init__()? I've written many exception classes, often subclasses of some other exception, but I've hardly ever paid attention to the superclass __init__(). It usually suffices to set the right attributes. Jeremy From jeremy@zope.com Sat Apr 6 05:40:34 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Sat, 6 Apr 2002 00:40:34 -0500 Subject: [Python-Dev] Namespaces In-Reply-To: <15534.34573.142067.134677@anthem.wooz.org> References: <20020401025823.GA29354@panix.com> <20020401185657.GA6094@panix.com> <15528.51794.642978.22967@slothrop.zope.com> <20020401212232.GA14508@panix.com> <20020406004705.GA751@panix.com> <15534.18931.480692.612031@slothrop.zope.com> <20020406015524.GA9543@panix.com> <15534.24071.152009.959329@anthem.wooz.org> <20020406043948.GA1681@panix.com> <15534.34573.142067.134677@anthem.wooz.org> Message-ID: <15534.35410.218532.485008@slothrop.zope.com> BUt does it really clarify anything to use binding to describe two different things? You bind a name. You set an attribute. Names are a static property of the program text (for the most part). Attributes are quite dynamic. The only place where these two meet is modules. I think that's exceptional rather than indicative of some deeper connection between the two concepts. Functions have names, but those names don't relate to its attributes. Jeremy From barry@zope.com Sat Apr 6 06:03:55 2002 From: barry@zope.com (Barry A. Warsaw) Date: Sat, 6 Apr 2002 01:03:55 -0500 Subject: [Python-Dev] Deprecating string exceptions References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <15534.14484.134938.791320@slothrop.zope.com> <200204060018.g360IKI22119@pcp742651pcs.reston01.va.comcast.net> <15534.18809.964868.250044@slothrop.zope.com> <200204060202.g36221e22571@pcp742651pcs.reston01.va.comcast.net> <15534.24294.725727.593378@anthem.wooz.org> <200204060316.g363GCG22664@pcp742651pcs.reston01.va.comcast.net> <15534.33568.340660.611621@anthem.wooz.org> <15534.35263.155509.971861@slothrop.zope.com> Message-ID: <15534.36811.27859.383515@anthem.wooz.org> >>>>> "JH" == Jeremy Hylton writes: JH> I'm not sure what the base class(es) actually buys us. We JH> could just as well say by fiat that the interpreter will set JH> the traceback attribute of an exception instance to the JH> traceback. Why do we need a special base class to accomplish JH> that? This is Python. You don't need to do isinstance(). JH> You just need to see if it has the right attributes. You might be right. It certainly hasn't hindered us up 'til now. JH> BTW, what is the signature for Exception.__init__()? I've JH> written many exception classes, often subclasses of some other JH> exception, but I've hardly ever paid attention to the JH> superclass __init__(). It usually suffices to set the right JH> attributes. When it was written in Python, I believe it was defined as class Exception: def __init__(self, *args): self.args = args and that's essentially what it is now that it's implemented in C. But many of the derived exception classes have their own funky __init__()s for backwards compatibility, e.g. EnvironmentError which looks at the number of arguments to decide which of the errno, strerror, and filename attributes to set. -Barry From martin@v.loewis.de Sat Apr 6 07:00:03 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 06 Apr 2002 09:00:03 +0200 Subject: [Python-Dev] Namespaces In-Reply-To: <20020406004705.GA751@panix.com> References: <002901c1d7f2$8b04f560$2eb53bd0@othello> <20020401025823.GA29354@panix.com> <20020401185657.GA6094@panix.com> <15528.51794.642978.22967@slothrop.zope.com> <20020401212232.GA14508@panix.com> <20020406004705.GA751@panix.com> Message-ID: Aahz writes: > Rebinding a does not affect the originally bound object > (unless the originally bound object's reference count goes to zero). > > Any ideas about what to call ? (Calling it a binding sounds a > little too self-referential.) That is a "name". > > Furthermore, some attributes live in multiple namespaces. Given > > > > obj.name > > > > what namespace is considered to find the name? NOT the namespace of > > obj, alone - Python also considers the namespace of obj's class (if > > obj is an instance), of the base classes, etc. OTOH, > > > > obj.name = None > > > > modifies the namespace of obj (unless name is a computed attribute). I'm saying that name lookup considers multiple namespaces in some cases: >>> class X: ... name = 1 ... >>> x.name 1 >>> x.__dict__.has_key("name") 0 So saying that x.name yields the value from searching the name in the namespace of x is wrong. Regards, Martin From martin@v.loewis.de Sat Apr 6 07:00:55 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 06 Apr 2002 09:00:55 +0200 Subject: [Python-Dev] Namespaces In-Reply-To: <20020406043948.GA1681@panix.com> References: <20020401025823.GA29354@panix.com> <20020401185657.GA6094@panix.com> <15528.51794.642978.22967@slothrop.zope.com> <20020401212232.GA14508@panix.com> <20020406004705.GA751@panix.com> <15534.18931.480692.612031@slothrop.zope.com> <20020406015524.GA9543@panix.com> <15534.24071.152009.959329@anthem.wooz.org> <20020406043948.GA1681@panix.com> Message-ID: Aahz writes: > What do we call all three of these collectively? I'd say that I agree > with MvL that computed bindings are not in fact names They are names - they just don't live in namespaces. Regards, Martin From tim.one@comcast.net Sat Apr 6 07:10:14 2002 From: tim.one@comcast.net (Tim Peters) Date: Sat, 06 Apr 2002 02:10:14 -0500 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) In-Reply-To: <15534.24411.659593.420529@grendel.zope.com> Message-ID: [Fred L. Drake, Jr.] > ... > The HTML Help browser is substantially better than browsing a bunch of > HTML files, primarily because of the index and full text search. Careful, there -- like all things Microsoft, there's HTML Help, and then there's HTML Help. It's quite possible Guido remembers a very early version, which sucked rotten eggs thru a Tabasco-coated nostril straw. Guido can see some screenshots of a modern version here: http://www.orgmf.com.ar/condor/pyshelfscreen.html And if that's interesting to him, he can download the whole PythonShelf by going up a level, or just the std doc set for 2.2 (pythlp.chm). One advantage is that the whole 2.2 doc set lives in a single 3MB file this way. As we distribute it, 2.2 docs comprised 1,016 files consuming well over 8MB unpacked. Irritations include that the set of control buttons aren't the same as in your browser, vary across .chm files, and there's often no obvious way for the user to change font size (wrt the pythlp.chm above, a persistent user may figure out that they can close the file, change their font size in IE, then reopen the .chm file). Amazingly, there's no help available to the user *about* HTML Help, and in particular there's not a clue about the power hiding in the search facility (which *may* include blazing fast wildcards, boolean expressions, and proximity search, depending how on clueless the .chm author was). > ... > The only (substantial) reason I've not been building the HTML Help > version is that I've not had the time to figure out how to do it. > I don't think it's all that difficult, but I need to spend a > little time with it and make sure I understand the tools. No, the right way to proceed with an MS tool is to give up on understanding, and just figure out which buttons you need to push in which order, by observing over time which button sequences didn't work -- I have a Windows box you can use, provided you can reboot it yourself . From nhodgson@bigpond.net.au Sat Apr 6 07:44:56 2002 From: nhodgson@bigpond.net.au (Neil Hodgson) Date: Sat, 6 Apr 2002 17:44:56 +1000 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) References: Message-ID: <04b801c1dd3e$f28c5b00$0acc8490@neil> Tim: > Careful, there -- like all things Microsoft, there's HTML Help, and then > there's HTML Help. And then there's the new Document Explorer which uses the new .HxS file format. Microsoft changes documentation format and viewer apps every couple of years or so to inflict a mixture of pain and wonder on us serfs. The .NET documentation comes in the new format and I have read of the problems (and hence avoided them myself) from people who have downloaded the latest platform SDK which has documentation in the new format and so will not load in the old viewer within MSVC 6. > Irritations include that the set of control buttons aren't the same as in > your browser, vary across .chm files, and there's often no obvious way for > the user to change font size (wrt the pythlp.chm above, a persistent user > may figure out that they can close the file, change their font size in IE, > then reopen the .chm file). Amazingly, there's no help available to the > user *about* HTML Help, and in particular there's not a clue about the power > hiding in the search facility (which *may* include blazing fast wildcards, > boolean expressions, and proximity search, depending how on clueless the > .chm author was). Most of this is fixed in Document Explorer although there are fiddly incompatibilities that annoy me. Neil From fredrik@pythonware.com Sat Apr 6 08:33:24 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Sat, 6 Apr 2002 10:33:24 +0200 Subject: [Python-Dev] Namespaces References: <002901c1d7f2$8b04f560$2eb53bd0@othello><20020401025823.GA29354@panix.com><20020401185657.GA6094@panix.com><15528.51794.642978.22967@slothrop.zope.com><20020401212232.GA14508@panix.com><20020406004705.GA751@panix.com><15534.18931.480692.612031@slothrop.zope.com><20020406015524.GA9543@panix.com> <15534.24071.152009.959329@anthem.wooz.org> Message-ID: <00da01c1dd45$bb996a00$ced241d5@hagrid> barry wrote: > I'm curious why setting an attribute isn't a name binding operation? > Are you binding the object to the attribute name in the object's > attribute namespace? Since that maps to a setting in an __dict__ > (usually), what /would/ you call it? syntactic sugar? A = 1 binds 1 to the name "A" A.B = 1 calls A.__setattr__("B", 1) A["B"] = 1 calls A.__setitem__("B", 1) it's pretty clear that the first form differs from the others, but what's the difference between the second and the third form? or are all three name binding operations? From fredrik@pythonware.com Sat Apr 6 08:17:32 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Sat, 6 Apr 2002 10:17:32 +0200 Subject: [Python-Dev] Pymalloc and backward compatibility References: <714DFA46B9BBD0119CD000805FC1F53B01B5B2A4@UKRUX002.rundc.uk.origin-it.com><20020405151519.GA4682@panix.com><200204051721.g35HL7P14646@pcp742651pcs.reston01.va.comcast.net><20020405172548.GA8114@panix.com> Message-ID: <00d901c1dd45$bb06c6f0$ced241d5@hagrid> martin wrote: > Aahz writes: > > > I must be missing something here. We're not talking > > about an end-user product, we're talking about a developer tool. What's > > wrong with requiring a Python download for someone who at this point > > wants to *upgrade* vim from the system default? > > Because requiring this software might violate company policies. For > example, Redhat cannot include more recent versions of Python in their > distribution 7.x. what martin says should be obvious, of course -- people use Python in many different environments, and most of them are quite different from your average python-dev'ers development box. what I don't really understand why this discussion keeps going. isn't it up to the VIM maintainers to decide what requirements their customers might be willing to live with? (if you want me to tell you what I really think, ask me to force PIL users to upgrade to the latest Python version if they want to use the latest PIL version... or post a marketing PEP ;-) From fredrik@pythonware.com Sat Apr 6 13:21:37 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Sat, 6 Apr 2002 15:21:37 +0200 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format References: <014301c1dbe5$bbbffdc0$6d94fea9@newmexico> <03b201c1dc15$7b0f2aa0$6d94fea9@newmexico> <03b301c1dc17$58cfb390$ced241d5@hagrid> <04d301c1dc37$e8a194a0$6d94fea9@newmexico> Message-ID: <035101c1dd6d$fd12dc50$ced241d5@hagrid> Samuele Pedroni wrote: > > do you need a PEP, or is it okay if I just post some code? > > Code please, see code is less controversial than PEPs... some code can be found via this link: http://online.effbot.org/2002_04_01_archive.htm#75103766 the pythondoc parser can handle all modules in the xmltoys kit, but it hasn't been tested with much external code. out- puts both (preliminary, rather simple-minded) XML and a truly minimalistic HTML variant. requires Python 2.2 at the moment; haven't tested it under Jython yet. see TODO entries in xmltoys/PythonDoc for a list of known issues. enjoy /F From gmcm@hypernet.com Sat Apr 6 13:57:13 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Sat, 6 Apr 2002 08:57:13 -0500 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) In-Reply-To: References: <15534.8065.891619.224874@grendel.zope.com> Message-ID: <3CAEB869.17326.52030D11@localhost> On 6 Apr 2002 at 10:14, John Machin wrote: [snip] > (2) In two application development shops with which > I am engaged, the main application runs on a large > Unix box, but the developers' and DBAs' workstations > are Windows (2000) PCs -- is this atypical? Not at all. (Don't know why it matters.) -- Gordon http://www.mcmillan-inc.com/ From guido@python.org Sat Apr 6 14:00:24 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 06 Apr 2002 09:00:24 -0500 Subject: [Python-Dev] Deprecating string exceptions In-Reply-To: Your message of "Sat, 06 Apr 2002 00:38:07 EST." <15534.35263.155509.971861@slothrop.zope.com> References: <200203271657.g2RGvqe09883@mira.informatik.hu-berlin.de> <15534.14484.134938.791320@slothrop.zope.com> <200204060018.g360IKI22119@pcp742651pcs.reston01.va.comcast.net> <15534.18809.964868.250044@slothrop.zope.com> <200204060202.g36221e22571@pcp742651pcs.reston01.va.comcast.net> <15534.24294.725727.593378@anthem.wooz.org> <200204060316.g363GCG22664@pcp742651pcs.reston01.va.comcast.net> <15534.33568.340660.611621@anthem.wooz.org> <15534.35263.155509.971861@slothrop.zope.com> Message-ID: <200204061401.g36E0UE27659@pcp742651pcs.reston01.va.comcast.net> > I'm not sure what the base class(es) actually buys us. We could just > as well say by fiat that the interpreter will set the traceback > attribute of an exception instance to the traceback. Why do we need a > special base class to accomplish that? This is Python. You don't > need to do isinstance(). You just need to see if it has the right > attributes. Fair enough, though having a common base class makes it easier to augment the functionality -- every exception class would automatically inherit any functionality provided by the base class. But here's the real reason (which I brought up when we had this discussion a few weeks ago but didn't remember completely last night): once we reach the nirvana where all exceptions derive from Exception, we can declare that except: is equivament to except Exception: and then, when we also store the traceback on the exception object, we can (eventually) get rid of sys.exc_info(). > BTW, what is the signature for Exception.__init__()? I've written > many exception classes, often subclasses of some other exception, but > I've hardly ever paid attention to the superclass __init__(). It > usually suffices to set the right attributes. It's been a documented recommended practice to inherit from Exception since the introduction of class exceptions. Why are you suddenly fighting this? --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Sat Apr 6 14:07:02 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Sat, 6 Apr 2002 09:07:02 -0500 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) In-Reply-To: <3CAEB869.17326.52030D11@localhost> References: <15534.8065.891619.224874@grendel.zope.com> <3CAEB869.17326.52030D11@localhost> Message-ID: <15535.262.592852.474843@grendel.zope.com> Gordon McMillan writes: > (Don't know why it matters.) His concern is that we may not be sufficiently interested in the Windows platform, which is what's on his desktop. This is not an unreasonable concern, given the appearant misunderstanding of Tim's meaning when he described our approach to the Windows port "reactive". -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From aahz@pythoncraft.com Sat Apr 6 14:22:59 2002 From: aahz@pythoncraft.com (Aahz) Date: Sat, 6 Apr 2002 09:22:59 -0500 Subject: [Python-Dev] Namespaces In-Reply-To: <15534.34573.142067.134677@anthem.wooz.org> References: <20020401185657.GA6094@panix.com> <15528.51794.642978.22967@slothrop.zope.com> <20020401212232.GA14508@panix.com> <20020406004705.GA751@panix.com> <15534.18931.480692.612031@slothrop.zope.com> <20020406015524.GA9543@panix.com> <15534.24071.152009.959329@anthem.wooz.org> <20020406043948.GA1681@panix.com> <15534.34573.142067.134677@anthem.wooz.org> Message-ID: <20020406142259.GA29158@panix.com> On Sat, Apr 06, 2002, Barry A. Warsaw wrote: > > >>>>> "A" == Aahz writes: > > A> * primary names (gotten from the lang ref), which are names in > A> the local/global/builtin namespace > > A> * object names (and a module is an object ;-) AKA attributes > > I'd just call these "attributes" . Okay, maybe "attribute names". I'd agree with that if I agreed with what you later say about bindings. > A> * computed bindings (getitem/setitem) > > I'd call these "computed attributes". I don't like "computed > bindings" because that sounds like you're saying that you're computing > both the target of the binding and the object that is being bound (or > one or the other?). Ah! "Target" is a good word. > A> What do we call all three of these collectively? I'd say that > A> I agree with MvL that computed bindings are not in fact names, > > "Computed attributes" aren't bindings, but that only matters if you > can tell the difference. In a normal situation, if you see "obj.foo" > it may not matter that "foo" only exists by virtue of a > setattr/getattr/descriptor. I'd still claim that "foo" is an > attribute name because it's part of the public interface for obj, > albeit perhaps a virtual attribute name. Oh, yes, they are bindings. Thinking about this a bit further, a binding is anything that increases the refcount of an object. If you don't want to use the word "binding" to describe this, come up with some other word that describes the same thing. > A> and as I said in my other post, using "binding" as the > A> collective noun looks ugly when we're also using it as a verb. > A> The key is to come up with some good way of saying > > A> When rebinding a ... > > Names get bound, so names get rebound. At least, that's how I think > about it. Right. And if I agree with MvL that computed bindings are names, then this is fine. Guido? Are you paying any attention to this discussion? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From aahz@pythoncraft.com Sat Apr 6 14:28:23 2002 From: aahz@pythoncraft.com (Aahz) Date: Sat, 6 Apr 2002 09:28:23 -0500 Subject: [Python-Dev] Namespaces In-Reply-To: <00da01c1dd45$bb996a00$ced241d5@hagrid> References: <15534.24071.152009.959329@anthem.wooz.org> <00da01c1dd45$bb996a00$ced241d5@hagrid> Message-ID: <20020406142823.GB29158@panix.com> On Sat, Apr 06, 2002, Fredrik Lundh wrote: > barry wrote: >> >> I'm curious why setting an attribute isn't a name binding operation? >> Are you binding the object to the attribute name in the object's >> attribute namespace? Since that maps to a setting in an __dict__ >> (usually), what /would/ you call it? > > syntactic sugar? > > A = 1 binds 1 to the name "A" > > A.B = 1 calls A.__setattr__("B", 1) > > A["B"] = 1 calls A.__setitem__("B", 1) > > it's pretty clear that the first form differs from the others, > but what's the difference between the second and the third > form? or are all three name binding operations? The way I'm currently thinking of this (thanks to Barry) is that all three are target binding operations. I'm currently leaning toward attributes being names because (like primaries) they go through namespace lookup on reads, but I'm open to counter-arguments. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From aahz@pythoncraft.com Sat Apr 6 14:31:48 2002 From: aahz@pythoncraft.com (Aahz) Date: Sat, 6 Apr 2002 09:31:48 -0500 Subject: [Python-Dev] Namespaces In-Reply-To: References: <002901c1d7f2$8b04f560$2eb53bd0@othello> <20020401025823.GA29354@panix.com> <20020401185657.GA6094@panix.com> <15528.51794.642978.22967@slothrop.zope.com> <20020401212232.GA14508@panix.com> <20020406004705.GA751@panix.com> Message-ID: <20020406143148.GC29158@panix.com> On Sat, Apr 06, 2002, Martin v. Loewis wrote: > Aahz writes: >> >> Rebinding a does not affect the originally bound object >> (unless the originally bound object's reference count goes to zero). >> >> Any ideas about what to call ? (Calling it a binding sounds a >> little too self-referential.) > > That is a "name". I'm willing to go along with that if others agree. >>> Furthermore, some attributes live in multiple namespaces. Given >>> >>> obj.name >>> >>> what namespace is considered to find the name? NOT the namespace of >>> obj, alone - Python also considers the namespace of obj's class (if >>> obj is an instance), of the base classes, etc. OTOH, >>> >>> obj.name = None >>> >>> modifies the namespace of obj (unless name is a computed attribute). > > I'm saying that name lookup considers multiple namespaces in some > cases: > >>>> class X: > ... name = 1 > ... >>>> x.name > 1 >>>> x.__dict__.has_key("name") > 0 You left out x=X()? > So saying that x.name yields the value from searching the name in the > namespace of x is wrong. Sure, no argument here. Right now, I'm trying to settle the issue of creating bindings before getting into the topic of namespace lookups. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From goodger@users.sourceforge.net Sat Apr 6 15:04:06 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Sat, 06 Apr 2002 10:04:06 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard Docstring Format In-Reply-To: <014801c1dd2a$1ece8360$6d94fea9@newmexico> Message-ID: David Abrahams wrote: > Is it just me, or are docstrings less-convenient than comments? How do you mean? > Any thought given to reST-processing a module's comments? HappyDoc does this, in addition to docstrings. Docutils hasn't gotten that far yet, and before it does I intend to analyze HappyDoc thoroughly. -- David Goodger goodger@users.sourceforge.net Open-source projects: - Python Docstring Processing System: http://docstring.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net - The Go Tools Project: http://gotools.sourceforge.net From guido@python.org Sat Apr 6 15:13:17 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 06 Apr 2002 10:13:17 -0500 Subject: [Python-Dev] Namespaces In-Reply-To: Your message of "Sat, 06 Apr 2002 09:28:23 EST." <20020406142823.GB29158@panix.com> References: <15534.24071.152009.959329@anthem.wooz.org> <00da01c1dd45$bb996a00$ced241d5@hagrid> <20020406142823.GB29158@panix.com> Message-ID: <200204061513.g36FDHF28547@pcp742651pcs.reston01.va.comcast.net> > > A = 1 binds 1 to the name "A" > > > > A.B = 1 calls A.__setattr__("B", 1) > > > > A["B"] = 1 calls A.__setitem__("B", 1) > > > > it's pretty clear that the first form differs from the others, > > but what's the difference between the second and the third > > form? or are all three name binding operations? I don't know if it matters for your explanation, but the *big* difference between A.B and A["B"] is that in the latter, A *is* the namespace, while in the former, A *has* a namespace (usually, but not always called A.__dict__). These namespaces are distinct, otherwise you'd have trouble keeping track of the difference between d["clear"] and d.clear... --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Sat Apr 6 15:48:19 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 06 Apr 2002 10:48:19 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Sat, 06 Apr 2002 16:29:52 +0200." References: <3CAEFC4E.2778AC55@python.org> Message-ID: <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> On c.p.py, Alex Martelli wrote: > Linus Thorvalds seems to have done pretty well with picking TWO > middle grounds -- two parallel tracks for Linux, "stable" and > "experimental". I'm moving the discussion to python-dev, since that's where I think it belongs. Personally, I hate the way Linux releases are numbered (I can never tell which one is stable and which one isn't). But I could get used to it if we used the micro version number to indicate stability -- in particular, 2.2 would be experimental, and 2.2.1 and following would be stable; 2.3 would be experimental, and 2.3.1 stable. This would require only a very slight change of policy: PEP 6 would have to be renamed from "Bug Fix Releases" to "Stable Releases". It wouldn't even have to loosen up its prohibitions; quoting: """ Prohibitions Patch releases are required to adhere to the following restrictions: 1. There must be zero syntax changes. All .pyc and .pyo files must work (no regeneration needed) with all patch releases forked off from a feature release. 2. There must be zero pickle changes. 3. There must be no incompatible C API changes. All extensions must continue to work without recompiling in all patch releases in the same fork as a feature release. Breaking any of these prohibitions requires a BDFL proclamation (and a prominent warning in the release notes). """ This doesn't prevent us to evolve the standard library. It doesn't prohibit adding new built-in functions, although I'd be reluctant to do so. It would, however, require a big change in how we present new releases to the world. Currently, Once 2.x is out, anything running 2.(x-1) is labeled obsolete, or at least oldfashioned. That needs to change! The website needs to present at least two options: - Bleeding edge (e.g. 2.3 alpha, or 2.3 once it's out) - Stable (e.g. 2.2.1) There may be more options: Zope 2 currently requires 2.1.2 and will require 2.1.3 as soon as it's out; other people still swear by 1.5.2. (I don't think that the 1.6 and 2.0 lines are still popular.) --Guido van Rossum (home page: http://www.python.org/~guido/) From aleax@aleax.it Sat Apr 6 16:06:21 2002 From: aleax@aleax.it (Alex Martelli) Date: Sat, 6 Apr 2002 18:06:21 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> References: <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> Message-ID: On Saturday 06 April 2002 05:48 pm, Guido van Rossum wrote: > On c.p.py, Alex Martelli wrote: > > Linus Thorvalds seems to have done pretty well with picking TWO > > middle grounds -- two parallel tracks for Linux, "stable" and > > "experimental". > > I'm moving the discussion to python-dev, since that's where I think it > belongs. Right. > Personally, I hate the way Linux releases are numbered (I can never > tell which one is stable and which one isn't). But I could get used > to it if we used the micro version number to indicate stability -- in > particular, 2.2 would be experimental, and 2.2.1 and following would > be stable; 2.3 would be experimental, and 2.3.1 stable. Sounds quite workable, as long as we never want to do "bug-fix" point releases for experimental/bleeding-edge versions. The Linux folks do several experimental A.B (with B being odd, meaning experimental).N, but then Linux kernel/modules/drivers ARE an order of magnitude larger than Python, so we may not have analogous needs -- I dunno. > This would require only a very slight change of policy: PEP 6 would > have to be renamed from "Bug Fix Releases" to "Stable Releases". It > wouldn't even have to loosen up its prohibitions; quoting: ... > This doesn't prevent us to evolve the standard library. It doesn't > prohibit adding new built-in functions, although I'd be reluctant to > do so. Yes, library changes should be OK, as long as they don't alter behavior of code which used to run (without specific BDFL proclamation and prominent notice). I personally wouldn't mind if some good-intention wording about not breaking existing doctests (e.g. no change in wording of error messages) snuck in, but then I'm a doctest junky. > It would, however, require a big change in how we present new releases > to the world. Currently, Once 2.x is out, anything running 2.(x-1) is > labeled obsolete, or at least oldfashioned. That needs to change! Yes, public perception issues are to some extent paramount to ensure this slight variation has the intended effect. Your freedom to experiment as well as public perception might be enhanced by some explicit mention that the first (.1) stable release does not guarantee 100% compatibility with the immediately previous bleeding-edge/experimental release, so you can (within some common-sense bounds) try putting something in 2.X and then taking it away in 2.X.1 if it didn't work. I.e., stability being guaranteed 2.X.1 -> 2.X.2 and so on, but not necessarily 2.X -> 2.X.1, just as not necessarily 2.(X-1).N -> 2..X.1. Other folks more adept at marketing than me may come up with other ways to help public perception of 2.X.Y as "good, solid, stable, NOT old, dusty, junky" and thus helping this variant on "dual-track" fulfil its role. Alex From David Abrahams" Message-ID: <049801c1dd8d$4da1c860$6401a8c0@boostconsulting.com> ----- Original Message ----- From: "David Goodger" > David Abrahams wrote: > > Is it just me, or are docstrings less-convenient than comments? > > How do you mean? I don't know, it's hard to put into words, but I'll try: 1. 5 more characters just to get started. Probably a shift key too, if I'm going to be stylistically conformant with other work I've seen. ("""..."""). 2. The docstring separates the function signature from its body, which tends to obscure the code a bit. I prefer prefix documentation. 3. Weird indentation when the docstring spans multiple lines def foo(bar, baz): """Start of doc rest of doc and some more doc""" function_body() Documentation is really hard to start with, and every additional barrier to entry is unacceptable to me. Every time I write a doc string, I think of all 3 of the above things, which causes a little "cognitive dissonance", and makes it less likely that I'll write another. -Dave From pedroni@inf.ethz.ch Sat Apr 6 17:31:25 2002 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Sat, 6 Apr 2002 19:31:25 +0200 Subject: [Python-Dev] PEP 287: reStructuredText Standard DocstringFormat References: Message-ID: <003001c1dd90$e0ba0100$6d94fea9@newmexico> From: David Goodger > David Abrahams wrote: > > Is it just me, or are docstrings less-convenient than comments? > > How do you mean? > >From my POV, docstrings want to be short, the problem being that they are placed inside definitions, they divide def from code. So the idea of a primary docstring (that end up in __doc__) and a secondary (only for auto-documentating extraction) docstring does not really solve this. Comments, naturally put in front of the definition, do not suffer of this. If I channel correctly that tradition, Common Lisp has docstrings too, but long docstrings are considered bad style. regards. From goodger@users.sourceforge.net Sat Apr 6 17:42:29 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Sat, 06 Apr 2002 12:42:29 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard DocstringFormat In-Reply-To: <049801c1dd8d$4da1c860$6401a8c0@boostconsulting.com> Message-ID: David Abrahams wrote: >>> Is it just me, or are docstrings less-convenient than comments? >> >> How do you mean? > > I don't know, it's hard to put into words, but I'll try: > > 1. 5 more characters just to get started. Probably a shift key too, if > I'm going to be stylistically conformant with other work I've seen. > ("""..."""). If the docstring is more than a few lines long, the quotes become average out to less than "#" in comments. Easier to edit docstrings; no per-line prefix. > 2. The docstring separates the function signature from its body, which > tends to obscure the code a bit. I prefer prefix documentation. I suppose that's valid. It comes down to what you're used to. You could put triple-quoted multi-line strings before the "def", but they wouldn't be accessible or particularly useful. Triple-quotes can be used as multi-line comments, at least for temporary ones. > 3. Weird indentation when the docstring spans multiple lines > > def foo(bar, baz): > """Start of doc > rest of doc > and some more doc""" > function_body() The convention is to indent all lines of the docstring equally, and to put the closing triple-quotes on a line by themselves. I prefer to put the opening triple-quotes on a line by themselves too. It would end up looking like this:: def foo(bar, baz): """ Start of doc rest of doc and some more doc """ function_body() > Documentation is really hard to start with, and every additional barrier > to entry is unacceptable to me. Every time I write a doc string, I think > of all 3 of the above things, which causes a little "cognitive > dissonance", and makes it less likely that I'll write another. You can still write prefix comments. HappyDoc extracts them, but they're not accessible from the running program or from the interactive interpreter. Better in the long run (for you and others who use your code) to embrace the features Python gives you. Confusing otherwise. Nobody's forcing you though. -- David Goodger goodger@users.sourceforge.net Open-source projects: - Python Docstring Processing System: http://docstring.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net - The Go Tools Project: http://gotools.sourceforge.net From pedroni@inf.ethz.ch Sat Apr 6 18:04:21 2002 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Sat, 6 Apr 2002 20:04:21 +0200 Subject: [Python-Dev] PEP 287: reStructuredText Standard DocstringFormat References: <003001c1dd90$e0ba0100$6d94fea9@newmexico> Message-ID: <008901c1dd95$7a72e420$6d94fea9@newmexico> From: Samuele Pedroni > > If I channel correctly that tradition, Common > Lisp has docstrings too, but long docstrings > are considered bad style. > My point is not against docstrings, it is against long primary/secondary docstrings, and at least IMHO the attitude of Guido wrt to leaving things as they are for the std lib seems to rhyme with this. Here is an example from Common Lisp: A docstring for the GETHASH function CL-USER 13 > (documentation 'gethash 'function) "Finds the entry in Hash-Table whose key is Key and returns the associated value and T as multiple values, or returns Default and Nil if there is no such entry." the full fledged doc of it: putting the markup for the second in the first would be an abuse. The primary/secondary splitting (not possible in CL), does not change this because *in the code*, they end up in the same place and count visually anyway as one *long* docstring. regards. From guido@python.org Sat Apr 6 18:47:26 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 06 Apr 2002 13:47:26 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard DocstringFormat In-Reply-To: Your message of "Sat, 06 Apr 2002 12:05:26 EST." <049801c1dd8d$4da1c860$6401a8c0@boostconsulting.com> References: <049801c1dd8d$4da1c860$6401a8c0@boostconsulting.com> Message-ID: <200204061847.g36IlQQ28731@pcp742651pcs.reston01.va.comcast.net> > 1. 5 more characters just to get started. Probably a shift key too, if > I'm going to be stylistically conformant with other work I've seen. > ("""..."""). Depends on how long your comment is going to be. For long comments, a on each line gets boring quickly, and not all editors know how to reformat such comment blocks right. > 2. The docstring separates the function signature from its body, which > tends to obscure the code a bit. I prefer prefix documentation. You have a small point there. > 3. Weird indentation when the docstring spans multiple lines > > def foo(bar, baz): > """Start of doc > rest of doc > and some more doc""" > function_body() Don't do this! There's an explicit rule that doc strings should be indented like the code containing them, and all docstring processors are supposed to compensate for this. Please write this, like everybody else does: def foo(bar, baz): """Start of doc rest of doc and some more doc""" function_body() Also note the blank line after the first line -- the first line is supposed to be a one-line summary of the function (for use in abbreviated help balloons, overviews, and so on). > Documentation is really hard to start with, and every additional > barrier to entry is unacceptable to me. Every time I write a doc > string, I think of all 3 of the above things, which causes a little > "cognitive dissonance", and makes it less likely that I'll write > another. Time for you to start reading more Python code -- I've never heard this excuse before. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Sat Apr 6 18:58:52 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 06 Apr 2002 13:58:52 -0500 Subject: [Python-Dev] PEP 287: reStructuredText Standard DocstringFormat In-Reply-To: Your message of "Sat, 06 Apr 2002 20:04:21 +0200." <008901c1dd95$7a72e420$6d94fea9@newmexico> References: <003001c1dd90$e0ba0100$6d94fea9@newmexico> <008901c1dd95$7a72e420$6d94fea9@newmexico> Message-ID: <200204061858.g36Iwqk28781@pcp742651pcs.reston01.va.comcast.net> > My point is not against docstrings, > it is against long primary/secondary docstrings, > and at least IMHO the attitude of Guido > wrt to leaving things as they are for the std > lib seems to rhyme with this. > > Here is an example from Common Lisp: > > A docstring for the GETHASH function > > CL-USER 13 > (documentation 'gethash 'function) > "Finds the entry in Hash-Table whose key is Key and returns the associated > value and T as multiple values, or returns Default and Nil if there is no > such entry." > > the full fledged doc of it: > thash> Indeed. This is how I originally envisioned docstrings, and this is how I still like to see them: short hints that mean you don't have to look it up in the manual in most cases (if you have some common sense and experience and aren't looking for how hairs are split in edge cases). > putting the markup for the second in the first would be an abuse. Agreed. > The primary/secondary splitting (not possible in CL), does > not change this because *in the code*, they end up in the same > place and count visually anyway as one *long* docstring. But note that there are other points of view. I don't mind if some package author wants to keep all the docs together with the code and wants to stick it all in the docstrings, and run some tool that extracts the docs and formats them as a reference manual. That's just not how I want to manage the standard library docs. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Sat Apr 6 19:08:20 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 06 Apr 2002 14:08:20 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Sat, 06 Apr 2002 18:06:21 +0200." References: <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204061908.g36J8KY28799@pcp742651pcs.reston01.va.comcast.net> > > Personally, I hate the way Linux releases are numbered (I can never > > tell which one is stable and which one isn't). But I could get used > > to it if we used the micro version number to indicate stability -- in > > particular, 2.2 would be experimental, and 2.2.1 and following would > > be stable; 2.3 would be experimental, and 2.3.1 stable. > > Sounds quite workable, as long as we never want to do "bug-fix" point > releases for experimental/bleeding-edge versions. The Linux folks do > several experimental A.B (with B being odd, meaning experimental).N, > but then Linux kernel/modules/drivers ARE an order of magnitude > larger than Python, so we may not have analogous needs -- I dunno. Or we could stay longer in beta. > Yes, library changes should be OK, as long as they don't alter behavior > of code which used to run (without specific BDFL proclamation and > prominent notice). I personally wouldn't mind if some good-intention > wording about not breaking existing doctests (e.g. no change in wording > of error messages) snuck in, but then I'm a doctest junky. I don't think so. Doctest is extremely picky, on purpose, and a breaking doctest doesn't mean that regular apps will break. > Yes, public perception issues are to some extent paramount to > ensure this slight variation has the intended effect. > > Your freedom to experiment as well as public perception might be > enhanced by some explicit mention that the first (.1) stable release > does not guarantee 100% compatibility with the immediately > previous bleeding-edge/experimental release, so you can (within > some common-sense bounds) try putting something in 2.X and > then taking it away in 2.X.1 if it didn't work. I.e., stability being > guaranteed 2.X.1 -> 2.X.2 and so on, but not necessarily 2.X -> 2.X.1, > just as not necessarily 2.(X-1).N -> 2..X.1. Good point. > Other folks more adept at marketing than me may come up with > other ways to help public perception of 2.X.Y as "good, solid, > stable, NOT old, dusty, junky" and thus helping this variant on > "dual-track" fulfil its role. Well, we can probably ride some of the common fear of X.0 releases. (Though we'd continuously have to explain that for Python, 2.X corresponds to X.0 in other vendors' numbering. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Sat Apr 6 19:41:43 2002 From: tim.one@comcast.net (Tim Peters) Date: Sat, 06 Apr 2002 14:41:43 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204061908.g36J8KY28799@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > Personally, I hate the way Linux releases are numbered (I can never > tell which one is stable and which one isn't). But I could get used > to it if we used the micro version number to indicate stability -- in > particular, 2.2 would be experimental, and 2.2.1 and following would > be stable; 2.3 would be experimental, and 2.3.1 stable. How are "experimental" and "stable" defined? 2.2 was as stable as we could make it with 7(!) full pre-releases strung out over half a year. 2.2.1 has fewer bugs, but it would be extremely optimistic to believe more surprises aren't lurking in, e.g., the type/class dark corners. So is 2.2.1 "stable"? Whatever that means, intuitively I doubt it will be *as* stable as 2.1.2. So long as we don't have people testing Python full-time (i.e., quite possibly forever), Python history says relatively few people will bother to try a pre-release, so lots of bugs have no hope of getting caught before an i.j.0 release. > ... > Or we could stay longer in beta. I don't think it would help much -- just a few days after initial release of an alpha or beta, downloads go way down. In practice, Python releases get a "street rep" that's not hard to pick up from c.l.py traffic. For example, several people independently recommend 2.1.2 as "the most stable" version of Python currently available, and I expect 2.2.1 will still be viewed as bleeding edge. Since there's no effective way to get wide testing of pre-releases (btw, I don't believe Linux shares this problem), there's no real way to judge a release's perceived stability until after it's been released. This makes a priori stability-number schemes "a problem" for us. Well, instead of calling 2.3 "2.3", we could call it 2.3.1. Then release 2.3.2, 2.3.3, ..., until consensus appears that 2.3.k is the most stable version of Python available. At that point we could re-release 2.3.k under the name 2.3 <0.9 wink>. microsoft-is-still-the-most-successful-software-vendor-in-the-solar- system-ly y'rs - tim From David Abrahams" <049801c1dd8d$4da1c860$6401a8c0@boostconsulting.com> <200204061847.g36IlQQ28731@pcp742651pcs.reston01.va.comcast.net> Message-ID: <04f101c1dda5$3f25aaa0$6401a8c0@boostconsulting.com> ----- Original Message ----- From: "Guido van Rossum" To: "David Abrahams" Cc: Sent: Saturday, April 06, 2002 1:47 PM Subject: Re: [Python-Dev] PEP 287: reStructuredText Standard DocstringFormat > > 1. 5 more characters just to get started. Probably a shift key too, if > > I'm going to be stylistically conformant with other work I've seen. > > ("""..."""). > > Depends on how long your comment is going to be. For long comments, a > on each line gets boring quickly, and not all editors know how to > reformat such comment blocks right. Ah, but /mine/ does <.002wink>. Anyway, I presume docstrings are supposed to be relatively short? Note that I am not comfortable just opening the docstring, because then the editor colors the rest of my program green until I add the closing quotes, so I have to type six quotes and three "move back"s just to get started. It takes nine boring lines of documentation to even approach the labor implied by opening comment characters. Any single-character-to-end-of-line syntax is going to make things easier. > > 2. The docstring separates the function signature from its body, which > > tends to obscure the code a bit. I prefer prefix documentation. > > You have a small point there. nay, it's bigger than a breadbox. > Don't do this! There's an explicit rule that doc strings should be > indented like the code containing them, and all docstring processors > are supposed to compensate for this. Please write this, like > everybody else does: > > def foo(bar, baz): > """Start of doc > > rest of doc > and some more doc""" > function_body() I assume what I'm seeing above is due to the fact that you used the hated tab key to indent part of the docstring? That wasn't what you intended, was it? Also, isn't the closing quote supposed to occupy a separate line? > Also note the blank line after the first line -- the first line is > supposed to be a one-line summary of the function (for use in > abbreviated help balloons, overviews, and so on). That's what I thought. Is there a guideline for how to write those, BTW? > > Documentation is really hard to start with, and every additional > > barrier to entry is unacceptable to me. Every time I write a doc > > string, I think of all 3 of the above things, which causes a little > > "cognitive dissonance", and makes it less likely that I'll write > > another. > > Time for you to start reading more Python code -- I've never heard > this excuse before. How is reading more Python code going to help? Not trying to be difficult, really. I'm willing to invest a little to train my stupid brain to stop complaining. Still wish I didn't have to, though. -Dave From aahz@pythoncraft.com Sat Apr 6 20:29:25 2002 From: aahz@pythoncraft.com (Aahz) Date: Sat, 6 Apr 2002 15:29:25 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> References: <3CAEFC4E.2778AC55@python.org> <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020406202925.GB14990@panix.com> On Sat, Apr 06, 2002, Guido van Rossum wrote: > > Personally, I hate the way Linux releases are numbered (I can never > tell which one is stable and which one isn't). But I could get used > to it if we used the micro version number to indicate stability -- in > particular, 2.2 would be experimental, and 2.2.1 and following would > be stable; 2.3 would be experimental, and 2.3.1 stable. I'm kind of in agreement with Tim; I don't think this buys much. Seems more important to me to specify process than stability per se. > It would, however, require a big change in how we present new releases > to the world. Currently, Once 2.x is out, anything running 2.(x-1) is > labeled obsolete, or at least oldfashioned. That needs to change! > The website needs to present at least two options: > > - Bleeding edge (e.g. 2.3 alpha, or 2.3 once it's out) > > - Stable (e.g. 2.2.1) > > There may be more options: Zope 2 currently requires 2.1.2 and will > require 2.1.3 as soon as it's out; other people still swear by 1.5.2. > (I don't think that the 1.6 and 2.0 lines are still popular.) What I suggest is that instead of major.minor.bugfix we do major.minor.enhanced.bugfix This way, only major.minor releases can introduce actual language changes, but enhancement releases can pick up library changes. If this sounds at all workable, I recommend issuing a quick BDFL pronouncement changing 2.2.1 to 2.2.0.1. Then what we might try for a 2.2.1 release is making the new boolean type a module that people can start using. I am also a bit amused because an early draft of PEP 6 did try to address this issue, but you and Tim dissuaded me. I think both of you were right back then, but it's still kind of funny to see it picked up a year later. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From tim.one@comcast.net Sat Apr 6 20:30:16 2002 From: tim.one@comcast.net (Tim Peters) Date: Sat, 06 Apr 2002 15:30:16 -0500 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) In-Reply-To: <15535.262.592852.474843@grendel.zope.com> Message-ID: [Fred L. Drake, Jr.] > His concern is that we may not be sufficiently interested in the > Windows platform, which is what's on his desktop. This is not an > unreasonable concern, given the appearant misunderstanding of Tim's > meaning when he described our approach to the Windows port > "reactive". I'm not sure John misunderstood my meaning badly, unless he read it as saying we ignore the Windows platform (which we certainly don't). We (PLabs) do very little in the way of *seeking* ways to improve Python on Windows specifically: we react to bug reports, and I sometimes sneak in a Windows "new feature" when a lack bites *me* in the butt (like adding os.waitpid() for Win32 in 2.3). But that's also true of Linux, and even more true of Macs than of Windows. The big difference is that Python's users contribute very little to improve Python on Windows specifically, compared to what they contribute for other OSes. So I don't object to calling our Windows-specific support "reactive" -- it overwhelmingly is, by any reasonable reading of the word. If John believes PLabs support of other OSes is not similarly reactive, though, then he's mistaken. I expect what he'd *like* to see is the kind of loving platform-specific support Jack Jansen contributes for the Mac. Mark Hammond is the only hope for that on Windows. Speaking of which, Mark's win32 extensions still aren't part of the core, and to any true Windows-head that probably appears as if PLabs "doesn't like" Windows (which isn't the case, but try explaining finite resources to a user ). From barry@zope.com Sat Apr 6 21:31:16 2002 From: barry@zope.com (Barry A. Warsaw) Date: Sat, 6 Apr 2002 16:31:16 -0500 Subject: [Python-Dev] Re: Stability and change References: <3CAEFC4E.2778AC55@python.org> <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> <20020406202925.GB14990@panix.com> Message-ID: <15535.26916.612116.365326@anthem.wooz.org> >>>>> "A" == Aahz writes: A> What I suggest is that instead of A> major.minor.bugfix A> we do A> major.minor.enhanced.bugfix A> This way, only major.minor releases can introduce actual A> language changes, but enhancement releases can pick up library A> changes. If this sounds at all workable, I recommend issuing a A> quick BDFL pronouncement changing 2.2.1 to 2.2.0.1. Ug, do we really need more dots? If we're really going to do that then maybe we should give up our biases about changing the first number. >From my own perspective it seems that 2.1.x is viewed as the stable release family, and each micro release reaffirms its stability. That's a good thing. That 2.2.x is viewed as more experimental is simply caused by the new type/class stuff, and I don't think there's much marketing you could do to change that perception. Maybe 2.2 should have been called 3.0 . -Barry From aahz@pythoncraft.com Sat Apr 6 21:37:14 2002 From: aahz@pythoncraft.com (Aahz) Date: Sat, 6 Apr 2002 16:37:14 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15535.26916.612116.365326@anthem.wooz.org> References: <3CAEFC4E.2778AC55@python.org> <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> <20020406202925.GB14990@panix.com> <15535.26916.612116.365326@anthem.wooz.org> Message-ID: <20020406213714.GA25387@panix.com> On Sat, Apr 06, 2002, Barry A. Warsaw wrote: > > From my own perspective it seems that 2.1.x is viewed as the stable > release family, and each micro release reaffirms its stability. > That's a good thing. That 2.2.x is viewed as more experimental is > simply caused by the new type/class stuff, and I don't think there's > much marketing you could do to change that perception. Maybe 2.2 > should have been called 3.0 . Hmph. Why are you winking? Between new-style classes and the division change, that's actually a sensible suggestion. How about we change "2.3" to "3.1", making 3.0 a retroactive symlink to 2.2. (Okay, so I'm mostly joking here, but I really do think there'd be fewer complaints about 2.2 if it had been billed as a major change.) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From pedroni@inf.ethz.ch Sat Apr 6 21:46:00 2002 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Sat, 6 Apr 2002 23:46:00 +0200 Subject: [Python-Dev] PEP 287: reStructuredText Standard DocstringFormat References: <003001c1dd90$e0ba0100$6d94fea9@newmexico> <008901c1dd95$7a72e420$6d94fea9@newmexico> <200204061858.g36Iwqk28781@pcp742651pcs.reston01.va.comcast.net> Message-ID: <001901c1ddb4$713d9e80$6d94fea9@newmexico> [Guido] > > The primary/secondary splitting (not possible in CL), does > > not change this because *in the code*, they end up in the same > > place and count visually anyway as one *long* docstring. > > But note that there are other points of view. I don't mind if some > package author wants to keep all the docs together with the code and > wants to stick it all in the docstrings, and run some tool that > extracts the docs and formats them as a reference manual. That's just > not how I want to manage the standard library docs. > Yup, but the final point is that easy to make happy also people that want to split the doc between the short informative docstring and a longish comment in front of the definition, and have the auto-doc extraction support this with reasonable options (extracting both things or just one). You can dislike this different approach, because the comment/docstring can be redundant or because, trying to avoid this, the reading vs. spatial orders of the two will not match. Personally I can see myself using this approach sometimes, it does not hurt my sensibility too much. It's very hard to come up with strong arguments wrt to these issues. working-for-a-peaceful-cohabitation-expressing-the-point- of-view-of-a-(maybe-1-person)-minority-ly y'rs, Samuele. From martin@v.loewis.de Sat Apr 6 21:44:55 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 06 Apr 2002 23:44:55 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> Message-ID: Alex Martelli writes: > Your freedom to experiment as well as public perception might be > enhanced by some explicit mention that the first (.1) stable release > does not guarantee 100% compatibility with the immediately > previous bleeding-edge/experimental release, so you can (within > some common-sense bounds) try putting something in 2.X and > then taking it away in 2.X.1 if it didn't work. I.e., stability being > guaranteed 2.X.1 -> 2.X.2 and so on, but not necessarily 2.X -> 2.X.1, > just as not necessarily 2.(X-1).N -> 2..X.1. Is that actually a problem (or, *the* problem)? I.e. if somebody finds that 2.2 is "unstable", does it ever happen that, to solve a problem, something has to be taken back? The typical problems seem more to be in the area of bugs: a bug was fixed, and this fix broke existing code, or a feature was introduced, and that broke existing code. In either case, it is desirable to keep the existing fix, and the new feature, but to find a way to let the old application continue to work. The need to phase out new features that turn out to be a mistake is relatively rare, IMO; instead, there is a stronger desire to phase out old features that have been superceded by newer ones. Or are you just hoping that booleans will be removed in 2.3.1 again :-? Regards, Martin From aahz@pythoncraft.com Sat Apr 6 22:08:21 2002 From: aahz@pythoncraft.com (Aahz) Date: Sat, 6 Apr 2002 17:08:21 -0500 Subject: [Python-Dev] Namespaces -> c.l.py Message-ID: <20020406220821.GA29044@panix.com> I'm moving the namespace discussion to comp.lang.python to get a broader range of input. I've just posted an incomplete draft of what I've got so far. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From barry@barrys-emacs.org Sat Apr 6 23:26:02 2002 From: barry@barrys-emacs.org (Barry Scott) Date: Sun, 7 Apr 2002 00:26:02 +0100 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Message-ID: <000301c1ddc2$6ad85180$070210ac@LAPDANCE> All that matters is that you have a numbering system and follow it consistently. The python users will figure out what the numbers mean to them. And I guess that 2.2 is not consistent. The changes from 1.5.2 to 2.0 seem to be closer to what happend with 2.1 to 2.2. One measure of the oddness of 2.2 is the lack of 3rd party support. BArry From hernan@orgmf.com.ar Sun Apr 7 01:12:45 2002 From: hernan@orgmf.com.ar (Hernan Martinez Foffani) Date: Sun, 7 Apr 2002 03:12:45 +0200 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) Message-ID: I'm not subscribed to the list, but just saw this when lurking through the archives. [Tim] >> If Fred volunteers to produce .chm files in time for a release, >> I'll ship 'em. Don't hold your breath . >> our-support-for-the-windows-distro-is-bug-driven-ly y'rs - tim I thought that Fred were now releasing them because I saw the link in python.org. [Fred] > .... The only (substantial) reason I've not been building the > HTML Help version is that I've not had the time to figure out > how to do it. I don't think it's all that difficult, but I need > to spend a little time with it and make sure I understand the tools. I'll be glad to help you whenever you tackle this. (email me) [John Machin] > (3) A script can be developed [Hernan presumably has at least the > foundation for this] to create the .chm file from the .html files > so that producing the .chm file at release time requires neither > volunteer nor conscript intervention ... The script and the procedure to build them are in web site: www.orgmf.com.ar/condor/pytstuff.html Running the script takes less than 15 minutes but... [Tim] > Possibly, but the idea that a new software process involving > many megabytes of data won't create new time sinks isn't credible > to me. Sure. For instance, I bet that one big nuisance for Fred is that the help compiler won't run under Unix so it can't be automated under the make docs. He will have to change OS or PC back and forth if he has to modify the latex sources, etc. Besides, the script parses the HTML files to pickup some relevant data (headers to build the TOC) so it can be sensitive to changes in the template and/or the latex2html program. --WRT win32 installers-- [Tim] > ... I doubt PLabs will ever produce an MSI installer. > ActiveState's installers have been better for some time already, > and I expect the gap to widen. The Windows user has many (I know of three) options to choose to install Python and all for free. Right now, PLabs' installer is pretty cool, I don't think we really need more fancy things here. BTW, if PLabs ever decide to add an HTML Help file a minor change will be needed in IDLE code. Thanks and regards, -Hernan From guido@python.org Sun Apr 7 01:18:45 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 06 Apr 2002 20:18:45 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Sat, 06 Apr 2002 15:29:25 EST." <20020406202925.GB14990@panix.com> References: <3CAEFC4E.2778AC55@python.org> <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> <20020406202925.GB14990@panix.com> Message-ID: <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> [Aahz] > I'm kind of in agreement with Tim; I don't think this buys much. Seems > more important to me to specify process than stability per se. I think our process is impeccable. What's missing is clear guidelines for the users. Maybe Alex is right and the Linux odd/even distinction is clearer than using micro release numbers to indicate bugfix/stable releases. > What I suggest is that instead of > > major.minor.bugfix > > we do > > major.minor.enhanced.bugfix Ah, yuck. I really don't want to start using more dots. Too confusing. > This way, only major.minor releases can introduce actual language > changes, but enhancement releases can pick up library changes. I don't see how this helps. > If this sounds at all workable, I recommend issuing a quick BDFL > pronouncement changing 2.2.1 to 2.2.0.1. No. :-) > Then what we might try for a 2.2.1 release is making the new boolean > type a module that people can start using. No again. (Banning the new bool type to a separate module is a waste of time.) > I am also a bit amused because an early draft of PEP 6 did try to > address this issue, but you and Tim dissuaded me. I think both of > you were right back then, but it's still kind of funny to see it > picked up a year later. We still seem to be disagreeing on how to do it. :-( [Barry] > From my own perspective it seems that 2.1.x is viewed as the stable > release family, and each micro release reaffirms its stability. That's about right. Maybe we should continue to promote 2.1.x and relegate 2.2 to the bleeding edge? A simple rearrangement of the website might be sufficient. > That's a good thing. That 2.2.x is viewed as more experimental is > simply caused by the new type/class stuff, and I don't think there's > much marketing you could do to change that perception. Maybe 2.2 > should have been called 3.0 . Or maybe 2.3 should become 2.2.3. <0.5 wink> [Martin] > [quoting Alex] > > I.e., stability being guaranteed 2.X.1 -> 2.X.2 and so on, but not > > necessarily 2.X -> 2.X.1, just as not necessarily 2.(X-1).N -> > > 2..X.1. > > Is that actually a problem (or, *the* problem)? I.e. if somebody > finds that 2.2 is "unstable", does it ever happen that, to solve a > problem, something has to be taken back? > > The typical problems seem more to be in the area of bugs: a bug was > fixed, and this fix broke existing code, or a feature was > introduced, and that broke existing code. Usually the latter. When fixing a bug breaks code, it can mean two things: (a) the fix was to introduce a feature (sometimes the right thing to do); or (b) the bug was so pervasive that work-arounds became widespread, and the fix breaks the work-around. Both are relatively rare (but have happened). But new features regularly break existing code: either by introducing a new keyword (e.g. yield), or by changing results in certain situations (e.g. str(1==1), or nested scopes), or by taking away old crutches (e.g. import * inside a function). There are many more examples, despite the fact that in each of these cases we've tried to do our best to minimize breakage. > In either case, it is desirable to keep the existing fix, and the > new feature, but to find a way to let the old application continue > to work. I would think that the best way to keep old code running is to keep the old interpreter around. Maybe one problem is that we've been installing each new version as "python" by default? Even on Windows that can be made to work, as long as you find a way to start your program with a specific Python version: you can't use #!/usr/local/bin/python2.2 and double clicking .py will invoke the most recently installed version. (Hm, would it be totally outrageous to let Python itself on Windows check for a #! line, and when there is one that names an existing Python interpreter that's not the current one, simply exec that one on top of the current one???) > The need to phase out new features that turn out to be a mistake is > relatively rare, IMO; instead, there is a stronger desire to phase > out old features that have been superceded by newer ones. IMO the most common breakage is when a new feature uses up a tiny portion of some namespace that used to belong to the application. > Or are you just hoping that booleans will be removed in 2.3.1 again > :-? Fat chance. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Sun Apr 7 01:52:55 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 06 Apr 2002 20:52:55 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Sat, 06 Apr 2002 20:18:45 EST." <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> References: <3CAEFC4E.2778AC55@python.org> <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204070152.g371qtq29107@pcp742651pcs.reston01.va.comcast.net> I just thought of another issue. James Logajan (possibly our most outspoken conservative developer) brought up the problem that whenever you install a new version, you start developing for that version, and your code is no longer portable to previous versions. Even if we had a perfect track record of keeping old code working, the addition of new, enticing features has a "lock-in" effect. (We may sometimes have done this on purpose, at least semi-consciously.) This would suggest designating each series 2.X, 2.X.1, 2.X.2, 2.X.3, ... as an asymptotic series approaching perfection for a given feature set, where the feature set is pretty much frozen the moment 2.X is released. Then user code portability within the 2.X series is close to guaranteed, making it easy to move a script to a previous version, and if that doesn't work because the script relies on some bug being fixed, there shouldn't be much resistance to upgrading 2.X.Y to 2.X.(Y+1). We have to commit to maintaining a 2.X series as long as it is in use. I don't think I want to chop off the leading "2." -- since as we're releasing new versions every 6-12 months, this would move us to 9.0 way too quickly. It shouldn't be too hard for users to get used to the fact that the jump from 2.X to 2.(X+1) is usually a big one (despite comments that the jump from 2.1 to 2.2 was as big as the one from 1.5.2 to 2.0 -- after all, so was the jump from 1.5.1 to 1.5.2, so we're already doing better. :-) I'm not sure what exactly we need to *do* now. Be clearer to the user community that we're still maintaining 2.1.*? Delay the 2.3 release and spend more time on 2.2.2 (assuming that 2.2.1 is about to go out)? Change the Makefile and the RPMs to install each 2.X version under a different name, e.g. only install as python2.2?[*] Rearrange the website to emphasize the stable version? While I agree that 2.1.x is the most stable release, there's also a significant user population who want to use 2.2 (e.g. the new Boost library wrappers use new-style classes). We should maintain the 2.2.x line for their benefit. And then maybe we could take our time releasing 2.3 while experimenting with new features to our heart's content. To counteract the fact (as Tim noted) that few users bother to download betas, maybe we could release 2.3 relatively rough, clearly mark it as an experimental release, and work on improvements and stability in 2.3.1, 2.3.2 and so on, until we're happy to call it stable and start experimenting with 2.4. While we're working the kinks out of 2.3.x, the most stable release advertised would be 2.1.x, and adventurous users could basically choose between 2.2.x (advanced but stable) or 2.3.x (bleeding edge). Compared to the Linux numbering scheme, we'd lose the "mnemonic" value (if you can call it that) of even-stable / odd-experimental; instead, we'd have to give each release a color, maybe starting with red (bleeding edge) going via yellow (advanced but stable) to green (most stable), and finally to blue (behind the pack) and black (dead). ----- [*] Here's an idea. Maybe "make install" should check if there's already a $(prefix)/bin/python that's got a different major.minor version number than the one it's about to install, and then *ask* whether to make a link named "python" to the new version rather than just doing it (unless you specified "make altinstall", which probably too few people know about). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Sun Apr 7 01:54:13 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 06 Apr 2002 20:54:13 -0500 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) In-Reply-To: Your message of "Sun, 07 Apr 2002 03:12:45 +0200." References: Message-ID: <200204070154.g371sIt29136@pcp742651pcs.reston01.va.comcast.net> > BTW, if PLabs ever decide to add an HTML Help file a minor change > will be needed in IDLE code. I'd be happy to take a patch. IDLE's got a few Windows-specific hacks to get to the right docs anyway. --Guido van Rossum (home page: http://www.python.org/~guido/) From nas@python.ca Sun Apr 7 03:07:25 2002 From: nas@python.ca (Neil Schemenauer) Date: Sat, 6 Apr 2002 18:07:25 -0800 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Sat, Apr 06, 2002 at 08:18:45PM -0500 References: <3CAEFC4E.2778AC55@python.org> <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020406180725.A14030@glacier.arctrix.com> Guido van Rossum wrote: > Maybe Alex is right and the Linux odd/even distinction is clearer than > using micro release numbers to indicate bugfix/stable releases. That has not been my impression. I have seen quite a few people confused by the Linux odd/even system. I think the current major.minor.micro system works just fine. I think the discussion should focus on whether more effort should be spent on micro releases. Perhaps 2.2.1 should have been finished before work was started on 2.3. Personally, I don't think so. Neil From aahz@pythoncraft.com Sun Apr 7 04:10:20 2002 From: aahz@pythoncraft.com (Aahz) Date: Sat, 6 Apr 2002 22:10:20 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204070152.g371qtq29107@pcp742651pcs.reston01.va.comcast.net> References: <3CAEFC4E.2778AC55@python.org> <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <200204070152.g371qtq29107@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020407031020.GA8668@panix.com> On Sat, Apr 06, 2002, Guido van Rossum wrote: > > I'm not sure what exactly we need to *do* now. Be clearer to the user > community that we're still maintaining 2.1.*? Delay the 2.3 release > and spend more time on 2.2.2 (assuming that 2.2.1 is about to go out)? > Change the Makefile and the RPMs to install each 2.X version under a > different name, e.g. only install as python2.2?[*] Rearrange the > website to emphasize the stable version? First of all, I agree with the idea of better management of multiple Python versions. But before we can start being clearer to the community, we need to be clearer to ourselves. Historically, there has been an attitude of, um, antipathetic neutrality toward the idea of maintaining old versions of Python. Until today, I was under the impression that your official attitude (as usually channeled by Tim) was that creating patch releases was something that PythonLabs would assist volunteers to accomplish, but would not push for in any way. It sounds to me like your attitude has in fact changed. If so, then I think the first step is to say so explicitly. After that, I'd like a clear statement of the new goalset for Python development. Once we have that, I think it should be relatively straightforward to create a process to support the new goals. Perhaps the whole PythonLabs crew should have an in-person brainstorming session to kickstart the goalmaking process? What's Zope doing to handle this issue? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From python@rcn.com Sun Apr 7 04:45:53 2002 From: python@rcn.com (Raymond Hettinger) Date: Sat, 6 Apr 2002 22:45:53 -0500 Subject: [Python-Dev] Re: Stability and change Message-ID: <000e01c1dde6$b8618560$98b53bd0@othello> This is a multi-part message in MIME format. ------=_NextPart_000_000B_01C1DDBC.CED2DBC0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable From: barry@zope.com (Barry A. Warsaw) > From my own perspective it seems that 2.1.x is viewed as the stable > release family, and each micro release reaffirms its stability. > That's a good thing. That 2.2.x is viewed as more experimental is > simply caused by the new type/class stuff, and I don't think there's > much marketing you could do to change that perception. Maybe 2.2 > should have been called 3.0 . Let's skip the dweebish and confusing 2.2.x in favor of plain English: 2.2 release 1 2.2 maintenance release 2 .... Raymond Hettinger, CPA ------=_NextPart_000_000B_01C1DDBC.CED2DBC0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
From: barry@zope.com (Barry A.=20 Warsaw)
> From my = own=20 perspective it seems that 2.1.x is viewed as the stable
> release = family,=20 and each micro release reaffirms its stability.
> That's a good=20 thing.  That 2.2.x is viewed as more experimental is
> simply = caused=20 by the new type/class stuff, and I don't think there's
> much = marketing=20 you could do to change that perception.  Maybe 2.2
> should = have been=20 called 3.0 <wink>.

Let's skip the dweebish and confusing = 2.2.x in=20 favor of plain English:
2.2 release 1
2.2 maintenance release 2
....
 
 
Raymond=20 Hettinger, CPA




 ------=_NextPart_000_000B_01C1DDBC.CED2DBC0-- From guido@python.org Sun Apr 7 05:11:27 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 06 Apr 2002 23:11:27 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Sat, 06 Apr 2002 22:10:20 EST." <20020407031020.GA8668@panix.com> References: <3CAEFC4E.2778AC55@python.org> <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <200204070152.g371qtq29107@pcp742651pcs.reston01.va.comcast.net> <20020407031020.GA8668@panix.com> Message-ID: <200204070411.g374BRW29273@pcp742651pcs.reston01.va.comcast.net> > Perhaps the whole PythonLabs crew should have an in-person > brainstorming session to kickstart the goalmaking process? Why should this be PythonLabs' internal brainstorm? We may be the only ones who get paid directly to maintain Python (part time, the rest is Zope work!), but we're not its only maintainers. This concerns us (active developers) all, because we're all limited resources. > What's Zope doing to handle this issue? Assuming you mean "how is change in the Zope software handled" (rather than "how does Zope Corporation handle change in Python"), they have a rather different approach. There's an effort to create a Zope 3 which is a total rewrite, (almost) completely from scratch. There's an explicit compatibility requirement for the end result, but that's only needed in the final phase -- while it's being created (with lots of help from the Zope community) it can do whatever it wants. In the mean time, various previous Zope releases are still being maintained: I believe that both 2.4 and 2.5 are under active maintenance, meaning that micro releases are occasionally issued, and there's an effort underway to create a 2.6 release (again with community help). While there's a striving for easy upgrading from 2.x to 2.(x+1), each new one (2.(x+1)) adds significant new features; I believe strict compatibility between 2.x and 2.(x+1) is not a goal (except at the storage level -- the database contents are sacred). I believe that it's very common for 3rd party products written against 2.x to require significant porting effort to work with 2.(x+1). The www.zope.org site for example runs a very old version of Zope (I believe 2.3) because it's encrusted with much custom code that make an upgrade difficult. Instead, there's a completely new version in the works (new.zope.org). --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Sun Apr 7 05:49:26 2002 From: tim.one@comcast.net (Tim Peters) Date: Sat, 06 Apr 2002 23:49:26 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > ... > (Hm, would it be totally outrageous to let Python itself on Windows > check for a #! line, and when there is one that names an existing > Python interpreter that's not the current one, simply exec that one on > top of the current one???) No, but how to name an existing interpreter is a problem: people don't install Python in the same locations, so an absolute path wouldn't be portable. At least for the PythonLabs and ActiveState distros, we could use the version number to look up the installation path in the Windows registry (the PLabs and AS installers create registry entries to record this info, under a key path with major.minor in its name, although the PLabs distro has never *used* this info; I believe the PythonWare distro doesn't set any info in the registry). From tim.one@comcast.net Sun Apr 7 06:16:05 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 07 Apr 2002 00:16:05 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204070152.g371qtq29107@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > I just thought of another issue. James Logajan (possibly our most > outspoken conservative developer) Ex-developer: he's made it quite clear that he no longer intends to use Python at all (after some brief current period of "compatibilizing" his old code). > brought up the problem that whenever you install a new version, you > start developing for that version, and your code is no longer portable > to previous versions. I must be exceptional -- this wasn't news to me . > Even if we had a perfect track record of keeping old code working, the > addition of new, enticing features has a "lock-in" effect. (We may > sometimes have done this on purpose, at least semi-consciously.) I've done it with full awareness in my own code. > This would suggest designating each series 2.X, 2.X.1, 2.X.2, 2.X.3, > ... as an asymptotic series approaching perfection for a given feature > set, where the feature set is pretty much frozen the moment 2.X is > released. If this differs from what we're doing today, I don't understand how: "no incompatibilities and new no features!" has been strongly observed in the bugfix releases. > Then user code portability within the 2.X series is close to guaranteed, > making it easy to move a script to a previous version, and if that > doesn't work because the script relies on some bug being fixed, there > shouldn't be much resistance to upgrading 2.X.Y to 2.X.(Y+1). We have > to commit to maintaining a 2.X series as long as it is in use. Again this *appears* to be arguing for what we already do. Maybe with one difference: I assure people on c.l.py that PLabs will continue distributing I.J.K for increasing K for as long as the community contributes the work for bugfix releases. Perhaps you're saying that PLabs should take that over? Note too that it wouldn't make James Logajan one whit happier, if we're to take him at his word. He developed some scripts for 1.5.2, and that's it. He wants them to work forever under all future Python releases and all platforms, and got particularly burned by Unicode's interaction with how Tk deals with UTF-8 (and a lot of people got burned by that). > ... > I'm not sure what exactly we need to *do* now. Be clearer to the user > community that we're still maintaining 2.1.*? Are we? We've made no effort to backport any bugfixes into 2.1.3, and to the contrary declared that we wouldn't do anything with 2.1.3 except fix the single gc-vs-trashcan disaster. We're not even building a Windows installer for 2.1.3 (last I heard). I don't think anyone is going to buy that as "maintaining" the 2.1.x series; I sure don't. > Delay the 2.3 release and spend more time on 2.2.2 (assuming that 2.2.1 > is about to go out)? 2.2 did go out the door unfinished . But, again, we could have delayed it two years and it still would have been relatively unstable: there's just no substitute for getting thousands of users *using* a release. > ... > While I agree that 2.1.x is the most stable release, there's also a > significant user population who want to use 2.2 (e.g. the new Boost > library wrappers use new-style classes). We should maintain the 2.2.x > line for their benefit. And then maybe we could take our time > releasing 2.3 while experimenting with new features to our heart's > content. To counteract the fact (as Tim noted) that few users bother > to download betas, maybe we could release 2.3 relatively rough, > clearly mark it as an experimental release, and work on improvements > and stability in 2.3.1, 2.3.2 and so on, until we're happy to call it > stable and start experimenting with 2.4. While we're working the > kinks out of 2.3.x, the most stable release advertised would be 2.1.x, > and adventurous users could basically choose between 2.2.x (advanced > but stable) or 2.3.x (bleeding edge). You can fiddle what you *call* what we're already doing anything you want, but if a long-lived two-branch comes out of this, I quit . I do believe we should direct more effort into bugfix releases; that can't be done without taking away from something else, though (that's a trivial deduction: everyone is already overloaded). From guido@python.org Sun Apr 7 06:37:01 2002 From: guido@python.org (Guido van Rossum) Date: Sun, 07 Apr 2002 00:37:01 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Sat, 06 Apr 2002 23:49:26 EST." References: Message-ID: <200204070537.g375b6m29326@pcp742651pcs.reston01.va.comcast.net> > [Guido] > > ... > > (Hm, would it be totally outrageous to let Python itself on Windows > > check for a #! line, and when there is one that names an existing > > Python interpreter that's not the current one, simply exec that one on > > top of the current one???) > > No, but how to name an existing interpreter is a problem: people > don't install Python in the same locations, so an absolute path > wouldn't be portable. At least for the PythonLabs and ActiveState > distros, we could use the version number to look up the installation > path in the Windows registry (the PLabs and AS installers create > registry entries to record this info, under a key path with > major.minor in its name, although the PLabs distro has never *used* > this info; I believe the PythonWare distro doesn't set any info in > the registry). Absolute paths aren't portable on Unix either, although #! /usr/bin/env python2.2 comes close. I wouldn't mind if users who need this feature would be required to know where their interpreter lives. But I wouldn't mind a registry-based solution either: #! python2.2 could mean "find Python 2.2 in the registry". I believe the PythonWare distro always installs in the same place, so if Fredrik thinks this will be useful, we could provide a hack that looks in certain places on the filesystem if no registry key is found. Still, I'd prefer to do the simplest thing that could possibly work, which IMO would be just an absolute pathname. It makes it *possible* for a specific script to request a specific interpreter. If you want fancy, you can create an installer that figures out the right path and sticks it in the #! line of scripts that need it at install time. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Sun Apr 7 06:58:02 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 07 Apr 2002 00:58:02 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204070537.g375b6m29326@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > Absolute paths aren't portable on Unix either, although > > #! /usr/bin/env python2.2 > > comes close. > > I wouldn't mind if users who need this feature would be required to > know where their interpreter lives. I don't believe any user needs this feature! Not on their own box. Let's be clear about "the problem" (or at least about Logajan's): you want to distribute your Python scripts all over the world. You're targeting Windows, and Unix flavors, and Macs (you hope). Freezing an interpreter into your distro isn't an option, as you have no idea how to even go about finding out how to do that on all the platforms your users run on, and you don't have access to all of them anyway. What then can you do to ensure that your scripts won't break on your *users'* boxes when they happen to upgrade their own Python installations? The only way I see that a "#!" line could do a lick of good for you then is if it were a x-platform way to identify version number, and that's all. It would also need to abort with "This app requires Python I.J" if it couldn't find Python I.J on the box. As a matter of fact, it wouldn't do Logajan any good even if Python 2.3 did a perfect job of that, as he only cares about 1.5.2. Going forward, maybe it would do someone some good. I'm skeptical of the cost/benefit ratio, though (James is extreme, IMO). > But I wouldn't mind a registry-based solution either: > > #! python2.2 > > could mean "find Python 2.2 in the registry". If it didn't also mean "find Python 2.2" on Linux and Macs, it wouldn't do anything to help the people who are unhappy in James's way. > I believe the PythonWare distro always installs in the same place, so > if Fredrik thinks this will be useful, we could provide a hack that > looks in certain places on the filesystem if no registry key is found. > > Still, I'd prefer to do the simplest thing that could possibly work, > which IMO would be just an absolute pathname. It can't possibly work, except to solve a problem I haven't heard anyone bring up. I expect the simplest thing that could possibly work for James's problem would be to add a pragma "#requires 2.3", and teach Python to complain if its own version doesn't match what the pragma says. It would also need some way to do this in .pyc and .pyo files. But that's not really want James wants either: he wants his scripts to run forever without change, under all Pythons that will ever be released, and on all platforms. From guido@python.org Sun Apr 7 08:09:35 2002 From: guido@python.org (Guido van Rossum) Date: Sun, 07 Apr 2002 03:09:35 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Sun, 07 Apr 2002 00:16:05 EST." References: Message-ID: <200204070709.g3779aJ32707@pcp742651pcs.reston01.va.comcast.net> > Again this *appears* to be arguing for what we already do. Maybe > with one difference: I assure people on c.l.py that PLabs will > continue distributing I.J.K for increasing K for as long as the > community contributes the work for bugfix releases. Perhaps you're > saying that PLabs should take that over? No, this is for all active developers. I agree that we don't have to change what we do very much, but I'm looking for a way to "spin" it to satisfy the more conservative user community (the ones who still like Python but are worried about the future). Since there are different "use cases", this may be hard or impossible, or we may have to explain it differently for different folks. For example, developers who distribute useful Python programs want to have a way to ensure that code they write gets run on a compatible Python version. We could make it easier to have different X.Y releases installed on the same system; this is already pretty easy on Unix if you know about "make altinstall". If we invent a standard way to specify the required version (or range of versions?) then that might be sufficient. You shot down my #! proposal for Windows, but maybe we can do something else. (Obviously this only works for programs, not for modules or packages. Package authors pretty much know the drill.) Beginning programmers have a different concern: they don't know much about software evolution, Python may be the first or second language they learn, and they hear that there's a concern about the rate of change. How can they know what that means? I've seen plenty of posts from such people worrying that in a few years from now they might have to learn a whole new language. They needn't worry, but it needs to be addressed with more explanation than "don't worry, the BDFL is watching over you." > > I'm not sure what exactly we need to *do* now. Be clearer to the > > user community that we're still maintaining 2.1.*? > > Are we? We've made no effort to backport any bugfixes into 2.1.3, > and to the contrary declared that we wouldn't do anything with 2.1.3 > except fix the single gc-vs-trashcan disaster. We're not even > building a Windows installer for 2.1.3 (last I heard). I don't > think anyone is going to buy that as "maintaining" the 2.1.x series; > I sure don't. Maybe we need to use a different word? Barry noted that 2.1.x is the most popular "stable" release and will probably remain that. So maybe we should say this is the "recommended" release. And then maybe we need to revise the decision not to make a Windows installer. Or maybe we *do* need to spend more effort backporting patches (not necessarily PLabs, but *somebody*). Another way to keep adding value to older releases is by backporting the build procedure enhancements. Once we've figured out how to build Python 2.3 on, say, some ancient version of AIX, maybe somebody should port that back to 2.2 and 2.1. That's not a language change, but still improves the usability of the release. (For some folks. Maybe the AIX user community is small enough that we shouldn't care, but there are other platforms with similar issues. I recall that you backported the new 32 bit Windows installer to 2.1.2; maybe the recent fixes could be backported to 2.1.3.) OK, so I *am* proposing to do something different. Maybe it's not very attractive to work on old releases. But Zope Corp does it all the time (and the fact that Zope 2.x requires Python 2.1.x may be an additional motivator). > > Delay the 2.3 release and spend more time on 2.2.2 (assuming that > > 2.2.1 is about to go out)? > > 2.2 did go out the door unfinished . But, again, we could > have delayed it two years and it still would have been relatively > unstable: there's just no substitute for getting thousands of users > *using* a release. Indeed. > > ... While I agree that 2.1.x is the most stable release, there's > > also a significant user population who want to use 2.2 (e.g. the > > new Boost library wrappers use new-style classes). We should > > maintain the 2.2.x line for their benefit. And then maybe we > > could take our time releasing 2.3 while experimenting with new > > features to our heart's content. To counteract the fact (as Tim > > noted) that few users bother to download betas, maybe we could > > release 2.3 relatively rough, clearly mark it as an experimental > > release, and work on improvements and stability in 2.3.1, 2.3.2 > > and so on, until we're happy to call it stable and start > > experimenting with 2.4. While we're working the kinks out of > > 2.3.x, the most stable release advertised would be 2.1.x, and > > adventurous users could basically choose between 2.2.x (advanced > > but stable) or 2.3.x (bleeding edge). > > You can fiddle what you *call* what we're already doing anything you > want, but if a long-lived two-branch comes out of this, I quit > . Would you settle for a separate CVS repository? :-) > I do believe we should direct more effort into bugfix releases; that > can't be done without taking away from something else, though > (that's a trivial deduction: everyone is already overloaded). Yup. That's why we need to put some effort in communication to the users, and some in a real change of priorities. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Sun Apr 7 08:53:55 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 07 Apr 2002 03:53:55 -0400 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar for Mozilla/Netscape) In-Reply-To: Message-ID: > [Fred] >> .... The only (substantial) reason I've not been building the >> HTML Help version is that I've not had the time to figure out >> how to do it. I don't think it's all that difficult, but I need >> to spend a little time with it and make sure I understand the tools. [Hernan Martinez Foffani] > I'll be glad to help you whenever you tackle this. (email me) Me too. I grabbed the MS Help Compiler and Robin Dunn's version of your script, and got a nice .chm for the distributed 2.2 docs after just one reboot . So I can walk Fred thru it in person now, incl. trivial changes (like setting the contents to show folders instead of "book" icons, and adding a Favorites tab, and telling the display window to remember its last size instead of opening too small to use each time - Fred will want all that). > ... > For instance, I bet that one big nuisance for Fred is that the help > compiler won't run under Unix so it can't be automated under the make > docs. He will have to change OS or PC back and forth if he has to > modify the latex sources, etc. Fred has a family, and all families run Windows in the US -- I think it may even be a law . I see that if nothing in the process screws up, going from Fred's HTML to a .chm goes very quickly (just a few minutes). > Besides, the script parses the HTML files to pickup some relevant > data (headers to build the TOC) so it can be sensitive to changes > in the template and/or the latex2html program. That's the part Fred needs to understand. I ignored all that: thanks! You made this very easy. > ... > The Windows user has many (I know of three) options to choose to > install Python and all for free. Right now, PLabs' installer is > pretty cool, I don't think we really need more fancy things here. MSI is especially important to corporate users, as it (at least in theory) allows admins to do mass remote installations and patch upgrades easily. The Wise 8.14 installer-builder has *some* sort of support for part of that too, but it appears it would take yet another near-total rewrite of the install script to exploit it. The part I'd be keenest to do (but won't do) is allowing for much smaller Windows patch releases. > BTW, if PLabs ever decide to add an HTML Help file a minor change > will be needed in IDLE code. Also to the installer (to fix the Python Docs shortcut), but these changes are dead easy. I'm all in favor of Fred making this possible . A caution: If it ever goes wrong in a deep way, we're hosed. MS says they'll never release another update to this system, and, indeed, the viewer component they make available on the HTML Help web site appears permanently obsolete now (IE 6 ships with a newer one). I expect this is just fallout from Neil Hodgson's: And then there's the new Document Explorer which uses the new .HxS file format. Microsoft changes documentation format and viewer apps every couple of years or so to inflict a mixture of pain and wonder on us serfs. From aleax@aleax.it Sun Apr 7 09:09:42 2002 From: aleax@aleax.it (Alex Martelli) Date: Sun, 7 Apr 2002 10:09:42 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> Message-ID: <02040710094202.05787@arthur> On Sunday 07 April 2002 03:18, Guido van Rossum wrote: ... > [Aahz] > > > I'm kind of in agreement with Tim; I don't think this buys much. Seems > > more important to me to specify process than stability per se. > > I think our process is impeccable. What's missing is clear guidelines > for the users. I agree with Guido. I think the issue is first one of perception by users and potential users of Python. I evangelize for Python on a regular basis, and I notice the issue of _perception_ of stability and change is a significant block for a certain important category of prospective users. Basically, users like my previous employer, think3, inc -- "previous" because, despite it being a great job on many other scores, I grew frustrated enough with being unable to evangelize Python there, that I left in February in order to be able to do Python full time. I had no problem firing up the enthusiasm of programmers and system administrators for using Python in scripting, testing and throw-away tasks -- Python "sells itself" in that way. But management wants to see a strategic picture, one with more perspective -- and it wants to see one of change and evolution within a framework of stability, before it starts OK'ing a given platform for programs with hundreds or thousands of function-points. We had a great opportunity when Microsoft announced the switch of Visual Basic from VB6 to VB.NET, with lots of enhancements that would break just about every existing program. The break with stability was so large, that even managers with a strong pro-MS bias were suddenly quite willing to consider alternatives. I think that if, at that point, I had been able to point at some implementation that was both stable _and_ evolving, I might have gotten the OK to do a pilot project. "All scripts written 2 years ago still run fine, see, but at the same time there is active ongoing maintenance -- bug fixes, new internal speed-ups, new stuff in the library, tools, etc, etc". I couldn't, and I couldn't even indicate such a perspective for the future -- each 2.N release would keep breaking some (little, but some) code of 2.(N-1), while 2.N.M would be constrained to absolutely no new functionality vs 2.N.(M-1), bug-fixes only, assuming there were any such 2.N.M. I'm not sure what they are considering now (besides C++, which dominates there but IS seen as hard and unproductive enough that the search for higher-productivity languages continues) -- maybe Java (that got a horrid reputation there due to its instability in '95/'96, but enough years have passed that they might look at it again now). "Keep the old interpreter around" is not the solution such prospective users want to hear. The choice between being locked off from _all_ enhancements OR having existing code break is what they have with Visual Basic, say -- exactly the unpleasant choice that makes them willing to look around for alternatives. "If you want the bright, new email package, or to say 'for key in somedict:', you also have to accept X and Y risks of existing code breakage -- oh and if you don't so accept it, we'd have to do our own port to the Sharp Zaurus because the existing one only works for 2.2, and ...", etc, etc. To some extent this reminds me of another similar frustration of mine, which was longstanding enough that I had more or less gotten used to it. We used to be a multi-platform shop, then switched to releasing our (mechanical CAD) apps only for Windows OS's, as the wide market started demanding the ability to use cheap, newly-powerful PCs instead of costly proprietary workstations. I kept proposing we do a Linux release as well, but each time the perception of "the platform changing under us" (even for the allegedly-stable 2.0.* series etc) was a big factori in blocking it. More recently I had had better luck with OpenBSD -- far stabler than Windows AND actively maintained -- to the point of doing some minor pilot projects for it. But in this case the problem was that our prospective user base had never heard of it (important when choosing an OS to target, though probably not a language for development) and would not be well placed to install and maintain it (probably true, it does require more skill than for Windows or one of the new, friendly Linux releases such as Mandrake or Suse). Still, the "stability _and_ change" paradigm did once again prove quite attractive to management -- they were at least willing to give OpenBSD a try, as they had not been for Linux. > Maybe Alex is right and the Linux odd/even distinction > is clearer than using micro release numbers to indicate bugfix/stable > releases. I'm not sure it's clearer. Clarity may not even matter all that much, as long as the message is heard as "there are two parallel lines of development: one along stable lines, that will break no previously correct code; one experimental, blazing new trails, for innovation and higher-risk changes" or something like that. What I could present today with integrity is more like "All core developers' interest focuses on the mildly-experimental track, constrained but only up to a point; oh yes, there _are_ sometimes bug fix releases to old and stabler versions, but absolutely no new stuff there -- all new things, even those which would break no previously correct code, always go into the mildly-experimental track". That's not the message the prospective users want to hear, I think. Not those I have in mind -- substantial software development shops considering whether to go ahead with Python for very substantial projects or to relegate it to testing/scripting/throw-away stuff but never functionality to be delivered to the customer. > > This way, only major.minor releases can introduce actual language > > changes, but enhancement releases can pick up library changes. > > I don't see how this helps. It would help to some degree if the "stable" releases could be shown to have SOME kind of enhancement. Not as much as if language changes which did not break existing correct code could also be there, but some. Modules such as email are a big part of the draw of 2.2, for example (not quite as big as language-level enhancements, but big). > > Then what we might try for a 2.2.1 release is making the new boolean > > type a module that people can start using. > > No again. (Banning the new bool type to a separate module is a waste > of time.) It would let people "return Boolean.True" where they want to do so in new parts of their programs while at the same time not worrying about str(x>y) changing under their feet. I.e., it would provide, or be perceived to provide, change within a framework of stability. Thus it would help us send the message I think prospective users want to hear -- that Python stands for stability AND change, that there is not necessarily a contradiction between the two goals. In a very minor way, to be sure. > > I am also a bit amused because an early draft of PEP 6 did try to > > address this issue, but you and Tim dissuaded me. I think both of > > you were right back then, but it's still kind of funny to see it > > picked up a year later. > > We still seem to be disagreeing on how to do it. :-( More relevantly, I'm not sure I had managed to explain exactly why I think "stability and change" IS important to Python adoption in a certain category of software shops. What numbering or naming scheme we adopt has importance in determining what we communicate, but before that, deciding what we DO want to communicate is probably more important. > [Barry] > > > From my own perspective it seems that 2.1.x is viewed as the stable > > release family, and each micro release reaffirms its stability. > > That's about right. Maybe we should continue to promote 2.1.x and > relegate 2.2 to the bleeding edge? A simple rearrangement of the > website might be sufficient. I think such a "retroactive" rebranding of 2.2 as "bleeding-edge" would be a serious blow to those of us who have accepted it as stable enough as our base over the last 3 months -- in books, magazine articles, advocacy within our firms and outside of them, etc. For example, AB Strakt has decided 2.2 was stable enough for us (and it was being presented as such) -- our new code now uses "for x in somedict:", "if x in somedict:", etc, for example -- now hearing that it wasn't, and 2.2.X micro-releases might introduce backwards-incompatible changes, would be a real problem indeed. > most recently installed version. (Hm, would it be totally outrageous > to let Python itself on Windows check for a #! line, and when there is > one that names an existing Python interpreter that's not the current > one, simply exec that one on top of the current one???) It would take away my ability to check if old code still works with a new release in a simple way -- by explicitly naming the new interpreter on the command line, whatever is in the shebang line in the script itself. > > Or are you just hoping that booleans will be removed in 2.3.1 again > > > > :-? > > Fat chance. :-) Indeed -- they won't go. So, stabilizing on 2.2.* (maybe with Booleans in a module, but surely no breakage of e.g. str(x>y)) might be one possibility, if the 2.2.* line was "stable and changing" - introducing enhancements but not ones that break previously correct code. I perceive Booleans as small (albeit negatively), but both our CTO and principal investor have vehemently posted against them, so it would be hard to 'sell' them even if I were quite favorable. Still, if that was the one hurdle to jump to get to a "stable and changing" release, who knows. But if jumping that small hurdle only gets to more of the same, either unstability or feature-frozen versions, the carrot may be too small. Alex From jafo-python-dev@tummy.com Sun Apr 7 11:20:38 2002 From: jafo-python-dev@tummy.com (Sean Reifschneider) Date: Sun, 7 Apr 2002 04:20:38 -0600 Subject: [Python-Dev] Pondering some changes to python.c... Message-ID: <20020407042038.O16962@tummy.com> I've been pondering some changes to the main() interpreter code for a while. I'd like to get some feedback on them. One is making changes to the process title to remove the interpreter name. I've implemented some code to do this, which I'll include below. I realize that this code won't work for every platform, but for some it would be quite a nice change. In particular, it would allow things like "killall" to be able to work on python programs. The basics are that I check to make sure all the argv elements are sequential, then just do "memcpy(argv[0], argv[1], appropriateSize)", and pad strlen(orig_argv[0]) at the end with '\0'. This is after saving a copy of the original argv to hand off to Python itself. The other thing I've been thinking of is a SUID python. I recently needed to make some code which ran SUID (so it could write files into a mail queue with the right ownership, no matter if it was coming in from the network or a local user). I ended up making my own copy of python.c which did the appropriate thing. However, it would seem like something that there should probably be an "official" one... It's just too hard to do it right otherwise... It would seem that if you were to unset LD_LIBRARY_PATH and PYTHONPATH (I'm probably missing something), and then pick up the priveleges specified in argv[1], that you could safely do SUID Python. Some folks I've mentioned it to seem to think it's just a bad idea to have an SUID python, but I think it's better to solve the problems once than have people re-inventing the wheel badly... I've had about 4 occasions where I needed to run something with particular privileges, and in at least one case my first cut at doing it left a fairly big hole (because I failed to unset LD_LIBRARY_PATH). Thoughts? Sean =========================== *** python.c.old Sun Apr 7 04:18:11 2002 --- python.c Sun Apr 7 04:19:32 2002 *************** *** 4,11 **** --- 4,71 ---- extern DL_EXPORT(int) Py_Main(int, char **); + #include + #include + #include + #include + + + int setprocstr(int argc, char ***argvIn) + { + char *endp, **argv = *argvIn; + int argsLen = 0, i; + + + /* do not change it if there is only one parameter */ + if (argc < 2) return(0); + + /* get max proc title length */ + for (endp = *argv, i = 0; i < argc; i++) { + endp += strlen(endp) + 1; + if (i == argc - 1) break; + + /* if arguments are not contiguous, abort */ + if ((endp) != argv[i + 1]) return(0); + } + argsLen = endp - argv[0]; + + /* preserve copy of args */ + { + char **tmpArgv; + int argvLen = sizeof(char *) * (argc + 1), offset; + + + /* allocate new argv */ + if (!(tmpArgv = malloc(argsLen + argvLen))) + return(0); + + /* copy over argument vectors */ + offset = argvLen; + for (i = 0; i < argc; i++) { + tmpArgv[i] = ((char *) tmpArgv) + offset; + offset += strlen(argv[i]) + 1; + } + tmpArgv[i] = NULL; + + /* copy over argument data */ + memcpy(tmpArgv[0], argv[0], argsLen); + + /* copy to passed-in pointer */ + *argvIn = tmpArgv; + } + + /* copy to new location and zero out remaining */ + memcpy(argv[0], argv[1], argsLen - (argv[1] - argv[0])); + memset(endp - (argv[1] - argv[0]), '\0', argv[1] - argv[0]); + argv[1] = NULL; + + return(1); + } + + int main(int argc, char **argv) { + setprocstr(argc, &argv); return Py_Main(argc, argv); } -- Q: What kind of dog goes "BOFH! BOFH!"? A: A rootweiler Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From fredrik@pythonware.com Sun Apr 7 12:38:05 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Sun, 7 Apr 2002 13:38:05 +0200 Subject: [Python-Dev] Pondering some changes to python.c... References: <20020407042038.O16962@tummy.com> Message-ID: <02d701c1de28$b13938e0$ced241d5@hagrid> Sean wrote: > The other thing I've been thinking of is a SUID python. I recently needed > to make some code which ran SUID (so it could write files into a mail > queue with the right ownership, no matter if it was coming in from the > network or a local user). I ended up making my own copy of python.c which > did the appropriate thing. However, it would seem like something that > there should probably be an "official" one... see /Misc/setuid-prog.c From skip@mojam.com Sun Apr 7 13:00:16 2002 From: skip@mojam.com (Skip Montanaro) Date: Sun, 7 Apr 2002 07:00:16 -0500 Subject: [Python-Dev] Weekly Python Bug/Patch Summary Message-ID: <200204071200.g37C0GH19992@12-248-41-177.client.attbi.com> Bug/Patch Summary ----------------- 222 open / 2377 total bugs (-18) 97 open / 1409 total patches (+2) New Bugs -------- SMTPLib sendmail fails on hyphenated adr (2002-03-31) http://python.org/sf/537484 packing double yields garbage (2002-04-02) http://python.org/sf/538361 Python open w/ MSVC6: bad error msgs (2002-04-03) http://python.org/sf/538827 Using the lib index mechanically (2002-04-03) http://python.org/sf/538961 resolver not thread safe (2002-04-04) http://python.org/sf/539175 asyncore file wrapper & os.error (2002-04-04) http://python.org/sf/539444 os.mkdir() handles SETGID inconsistently (2002-04-05) http://python.org/sf/539942 Framework.mainloop() - multiple threads (2002-04-05) http://python.org/sf/539990 New Patches ----------- ae* modules: handle type inheritance (2002-04-02) http://python.org/sf/538395 Support PyChecker in IDLE (2002-04-03) http://python.org/sf/539043 Webbrowser.py and konqueror (2002-04-04) http://python.org/sf/539360 Unicode fix for test in tkFileDialog.py (2002-04-04) http://python.org/sf/539392 build info docs from sources (2002-04-04) http://python.org/sf/539486 build info docs from tex sources (2002-04-04) http://python.org/sf/539487 dict.popitem(key=None) (2002-04-05) http://python.org/sf/539949 Remove PyMalloc_* symbols (2002-04-06) http://python.org/sf/540394 Closed Bugs ----------- readline() of codecs.StreamReader doesn't work for"utf-16le" (2000-11-14) http://python.org/sf/222395 popen('python -c"...."') tends to hang (2000-12-24) http://python.org/sf/226766 popen on Win9x isnt smart enough about finding w9xpopen.exe (2001-01-13) http://python.org/sf/228685 urllib2: AbstractHTTPHandler limits flexible client implemen (2001-03-11) http://python.org/sf/407783 popenx() argument mangling hangs python (2001-08-31) http://python.org/sf/457466 test_commands fails on SGI (2001-09-11) http://python.org/sf/460613 Integrate "What's new in Python x.y" (2001-12-26) http://python.org/sf/496922 test_commands assumes ls is in /bin (2001-12-27) http://python.org/sf/497160 Short-cuts missing for All Users (2001-12-30) http://python.org/sf/497854 urllib.py: open_http() host problem (2002-01-13) http://python.org/sf/503031 pprint should (optionally) use str (2002-01-17) http://python.org/sf/505152 array module has undocumented features (2002-02-17) http://python.org/sf/518767 posixmodule.c fails Tru64 (stat macro) (2002-03-01) http://python.org/sf/524600 tty shipped in win32 distribution (2002-03-03) http://python.org/sf/525121 Potential AV in vgetargskeywords (2002-03-24) http://python.org/sf/534347 new.instance() breaks with new classes (2002-03-26) http://python.org/sf/535170 super() broken with classmethods (2002-03-26) http://python.org/sf/535444 xml.sax memory leak with ExpatParser (2002-03-26) http://python.org/sf/535474 threading.Lock().acquire bug (2002-03-27) http://python.org/sf/536017 pydoc getopt fails (2002-03-28) http://python.org/sf/536449 module array: C type descriptions wrong (2002-03-28) http://python.org/sf/536469 Docs for "es" and "et" confusing (2002-03-28) http://python.org/sf/536516 Closed Patches -------------- socketmodule ssl: server & thread (2001-08-17) http://python.org/sf/452110 On the update_slot() behavior (2002-02-07) http://python.org/sf/514662 Explict proxies for urllib.urlopen() (2002-02-27) http://python.org/sf/523415 PEP 285 - Adding a bool type (2002-03-09) http://python.org/sf/528022 Comprehensibility patch (typeobject.c) (2002-03-28) http://python.org/sf/536407 pymalloc for types and other cleanups (2002-03-29) http://python.org/sf/536909 From gmcm@hypernet.com Sun Apr 7 14:55:24 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Sun, 7 Apr 2002 09:55:24 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> References: Your message of "Sat, 06 Apr 2002 15:29:25 EST." <20020406202925.GB14990@panix.com> Message-ID: <3CB0178C.28107.5727BDC2@localhost> On 6 Apr 2002 at 20:18, Guido van Rossum wrote: > ... When fixing a bug breaks code, it can mean two > things: (a) the fix was to introduce a feature > (sometimes the right thing to do); or (b) the bug > was so pervasive that work-arounds became > widespread, and the fix breaks the work-around. > Both are relatively rare (but have happened). I keep a fairly large body of code working with 1.5.2 onwards. If you broaden (b) to include cases where code that *accidentally* worked in an earlier release fails in a later release, then it covers 100% of the changes I've had to make. (1.5.2 to 2.0 was painful; the others have been easy). (FWIW, the hardest post 1.5.2 feature for me to do without is augmented assignment.) -- Gordon http://www.mcmillan-inc.com/ From ark@research.att.com Sun Apr 7 15:07:03 2002 From: ark@research.att.com (Andrew Koenig) Date: 07 Apr 2002 10:07:03 -0400 Subject: [Python-Dev] Pondering some changes to python.c... In-Reply-To: <20020407042038.O16962@tummy.com> References: <20020407042038.O16962@tummy.com> Message-ID: Sean> It would seem that if you were to unset LD_LIBRARY_PATH and Sean> PYTHONPATH (I'm probably missing something), and then pick up Sean> the priveleges specified in argv[1], that you could safely do Sean> SUID Python. Some folks I've mentioned it to seem to think it's Sean> just a bad idea to have an SUID python, but I think it's better Sean> to solve the problems once than have people re-inventing the Sean> wheel badly... You might want to be careful about LD_LIBRARY_PATH -- if the executable is built for dynamic linking, and it needs a library that's not in /usr/lib, mightn't changing LD_LIBRARY_PATH cause it to fail? -- Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark From guido@python.org Sun Apr 7 15:20:52 2002 From: guido@python.org (Guido van Rossum) Date: Sun, 07 Apr 2002 10:20:52 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Sun, 07 Apr 2002 09:55:24 EDT." <3CB0178C.28107.5727BDC2@localhost> References: Your message of "Sat, 06 Apr 2002 15:29:25 EST." <20020406202925.GB14990@panix.com> <3CB0178C.28107.5727BDC2@localhost> Message-ID: <200204071420.g37EKq008233@pcp742651pcs.reston01.va.comcast.net> > On 6 Apr 2002 at 20:18, Guido van Rossum wrote: > > ... When fixing a bug breaks code, it can mean two > > things: (a) the fix was to introduce a feature > > (sometimes the right thing to do); or (b) the bug > > was so pervasive that work-arounds became > > widespread, and the fix breaks the work-around. > > Both are relatively rare (but have happened). [Gordon] > I keep a fairly large body of code working with > 1.5.2 onwards. If you broaden (b) to include cases > where code that *accidentally* worked in an earlier > release fails in a later release, then it covers 100% > of the changes I've had to make. (1.5.2 to 2.0 was > painful; the others have been easy). Interesting. Two questions. (1) Got any details on which changes caused the most pain? (2) Was the pain worth it, or would you prefer we'd spent more time on being more backwards compatible? > (FWIW, the hardest post 1.5.2 feature for me > to do without is augmented assignment.) Since you're also a C programmer (I believe), I'm not surprised. --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik@pythonware.com Sun Apr 7 15:26:10 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Sun, 7 Apr 2002 16:26:10 +0200 Subject: [Python-Dev] Re: Stability and change References: Your message of "Sat, 06 Apr 2002 15:29:25 EST." <20020406202925.GB14990@panix.com> <3CB0178C.28107.5727BDC2@localhost> <200204071420.g37EKq008233@pcp742651pcs.reston01.va.comcast.net> Message-ID: <045f01c1de40$2c19dc10$ced241d5@hagrid> guido wrote: > (2) Was the pain worth it, or would you prefer we'd spent more time on > being more backwards compatible? fwiw, I think it would be a really good idea to decouple (large portions of) the standard library from the interpreter core, and keep the same code base running on multiple versions. if not else, it would force the core developers to "share the pain" ;-) From guido@python.org Sun Apr 7 15:34:11 2002 From: guido@python.org (Guido van Rossum) Date: Sun, 07 Apr 2002 10:34:11 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Sun, 07 Apr 2002 16:26:10 +0200." <045f01c1de40$2c19dc10$ced241d5@hagrid> References: Your message of "Sat, 06 Apr 2002 15:29:25 EST." <20020406202925.GB14990@panix.com> <3CB0178C.28107.5727BDC2@localhost> <200204071420.g37EKq008233@pcp742651pcs.reston01.va.comcast.net> <045f01c1de40$2c19dc10$ced241d5@hagrid> Message-ID: <200204071434.g37EYGZ08301@pcp742651pcs.reston01.va.comcast.net> > > (2) Was the pain worth it, or would you prefer we'd spent more time on > > being more backwards compatible? > > fwiw, I think it would be a really good idea to decouple (large > portions of) the standard library from the interpreter core, and > keep the same code base running on multiple versions. > > if not else, it would force the core developers to "share the > pain" ;-) I thought Gordon was saying that most of the pain was to fix code that *accidentally* worked in 1.5.2 but broke in 2.0. Whether or not you fix that in a backwards-compatible way, we've all shared that pain already. Other than sharing the pain, I don't see much of an advantage. --Guido van Rossum (home page: http://www.python.org/~guido/) From hernan@orgmf.com.ar Sun Apr 7 15:48:30 2002 From: hernan@orgmf.com.ar (Hern=?ISO-8859-1?Q?=E1n_Mart=ED?=nez Foffani) Date: Sun, 7 Apr 2002 10:48:30 -0400 Subject: [Python-Dev] IDLE patch (was: Searching Python docs) Message-ID: <200204071048.AA198115504@mail.orgmf.com.ar> [Guido] > [me] >> BTW, if PLabs ever decide to add an HTML Help file a minor change >> will be needed in IDLE code. > >I'd be happy to take a patch. IDLE's got a few Windows-specific hacks >to get to the right docs anyway. http://python.org/sf/540583 Regards, -Hernan From barry@zope.com Sun Apr 7 17:26:01 2002 From: barry@zope.com (Barry A. Warsaw) Date: Sun, 7 Apr 2002 12:26:01 -0400 Subject: [Python-Dev] Pondering some changes to python.c... References: <20020407042038.O16962@tummy.com> Message-ID: <15536.29465.372159.281895@anthem.wooz.org> >>>>> "AK" == Andrew Koenig writes: Sean> It would seem that if you were to unset LD_LIBRARY_PATH and Sean> PYTHONPATH (I'm probably missing something), and then pick Sean> up the priveleges specified in argv[1], that you could Sean> safely do SUID Python. Some folks I've mentioned it to seem Sean> to think it's just a bad idea to have an SUID python, but I Sean> think it's better to solve the problems once than have Sean> people re-inventing the wheel badly... AK> You might want to be careful about LD_LIBRARY_PATH -- if the AK> executable is built for dynamic linking, and it needs a AK> library that's not in /usr/lib, mightn't changing AK> LD_LIBRARY_PATH cause it to fail? It might indeed, although some *nixes have ways for the sysadmin to safely extend the default lookup path (i.e. /etc/ld.so.conf and ldconfig). -Barry From bh@intevation.de Sun Apr 7 20:04:46 2002 From: bh@intevation.de (Bernhard Herzog) Date: 07 Apr 2002 21:04:46 +0200 Subject: [Python-Dev] Pondering some changes to python.c... In-Reply-To: References: <20020407042038.O16962@tummy.com> Message-ID: <6qk7rjib3l.fsf@abnoba.intevation.de> Andrew Koenig writes: > You might want to be careful about LD_LIBRARY_PATH -- if the executable > is built for dynamic linking, and it needs a library that's not in > /usr/lib, mightn't changing LD_LIBRARY_PATH cause it to fail? It's likely that LD_LIBRARY_PATH won't work at all for setuid/setguid binaries. On Linux at least, ld.so ignores LD_LIBRARY_PATH for such executables. If it didn't you'd be able to substitute libc and other libraries a dynamically linked setuid executable uses which would be quite a huge security hole. I'm not sure how this is handled on other systems, though. Bernhard -- Intevation GmbH http://intevation.de/ Sketch http://sketch.sourceforge.net/ MapIt! http://www.mapit.de/ From gmcm@hypernet.com Sun Apr 7 21:38:18 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Sun, 7 Apr 2002 16:38:18 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204071420.g37EKq008233@pcp742651pcs.reston01.va.comcast.net> References: Your message of "Sun, 07 Apr 2002 09:55:24 EDT." <3CB0178C.28107.5727BDC2@localhost> Message-ID: <3CB075FA.17238.58989C59@localhost> On 7 Apr 2002 at 10:20, Guido van Rossum wrote: > [Gordon] > > I keep a fairly large body of code working with > > 1.5.2 onwards. [Guido] > Interesting. Two questions. > > (1) Got any details on which changes caused the most > pain? Tightening up functions which allowed two params where a tuple was correct. The changes to ConfigParser bit me hard; I think a couple other std lib changes got me, too. In many cases my 1.5.2 code ended up better, so the urge to whine is over pretty quickly. In a broader sense, Unicode is by far the most disruptive change. My excuses for ignoring the damn stuff are disappearing. > (2) Was the pain worth it, or would you prefer we'd > spent more time on > being more backwards compatible? I don't have more than a muted grumble about backwards compatibility. Where I end up with checking the version, it's to make use of a new feature, not keep old code working. Recompiling all those extensions is the biggest pain. > > (FWIW, the hardest post 1.5.2 feature for me > > to do without is augmented assignment.) > > Since you're also a C programmer (I believe), I'm > not surprised. Well, the other side of that coin is that I'm still only +0 on list comprehensions and -0 on lexical scoping :-). -- Gordon http://www.mcmillan-inc.com/ From guido@python.org Sun Apr 7 22:19:39 2002 From: guido@python.org (Guido van Rossum) Date: Sun, 07 Apr 2002 17:19:39 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Sun, 07 Apr 2002 16:38:18 EDT." <3CB075FA.17238.58989C59@localhost> References: Your message of "Sun, 07 Apr 2002 09:55:24 EDT." <3CB0178C.28107.5727BDC2@localhost> <3CB075FA.17238.58989C59@localhost> Message-ID: <200204072119.g37LJiV08575@pcp742651pcs.reston01.va.comcast.net> > Tightening up functions which allowed > two params where a tuple was correct. Yeah, we should have used warnings for that first. > The changes to ConfigParser bit me hard; I think we were too careless with ConfigParser. > I think a couple other std lib changes got > me, too. > > In many cases my 1.5.2 code ended up > better, so the urge to whine is over > pretty quickly. > > In a broader sense, Unicode is by far > the most disruptive change. My excuses > for ignoring the damn stuff are disappearing. Yeah, Unicode will continue to bite where you least expect it. :-( > > (2) Was the pain worth it, or would you prefer we'd > > spent more time on > > being more backwards compatible? > > I don't have more than a muted grumble > about backwards compatibility. Where I end > up with checking the version, it's to make use > of a new feature, not keep old code working. > > Recompiling all those extensions is the > biggest pain. Distutils to the rescue? > > > (FWIW, the hardest post 1.5.2 feature for me > > > to do without is augmented assignment.) > > > > Since you're also a C programmer (I believe), I'm > > not surprised. > > Well, the other side of that coin is that I'm > still only +0 on list comprehensions and -0 on > lexical scoping :-). That's OK. List comprehensions didn't incur any incompatibilities, and lexical scoping incurred only very rare ones (I think). --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik@pythonware.com Sun Apr 7 22:25:57 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Sun, 7 Apr 2002 23:25:57 +0200 Subject: [Python-Dev] PEP 287: reStructuredText Docstring Format References: <014301c1dbe5$bbbffdc0$6d94fea9@newmexico> <03b201c1dc15$7b0f2aa0$6d94fea9@newmexico> <03b301c1dc17$58cfb390$ced241d5@hagrid> <04d301c1dc37$e8a194a0$6d94fea9@newmexico> <035101c1dd6d$fd12dc50$ced241d5@hagrid> Message-ID: <06bb01c1de7a$d0df3530$ced241d5@hagrid> I wrote: > > Code please, see code is less controversial than PEPs... > > some code can be found via this link: > > http://online.effbot.org/2002_04_01_archive.htm#75103766 > > the pythondoc parser can handle all modules in the xmltoys > kit, but it hasn't been tested with much external code. out- > puts both (preliminary, rather simple-minded) XML and a truly > minimalistic HTML variant. an updated version can be found at: http://effbot.org/downloads/index.cgi/xmltoys-1.0a4-20020407.zip (click on download to get the code, click on the .html links to see examples rendered by the current HTML renderer) http://effbot.org/guides/pythondoc.htm (overview description, in progress) this version has been tested with itself, PIL, and a couple of other rather large libraries. unlike the first release, it can also parse the python standard library without getting stuck on None objects (but for obvious reasons, it doesn't generate any output for the standard library... not yet, at least ;-) it still requires Python 2.2; I'll fix the HTMLParser dependecy really soon now... enjoy /F From niemeyer@conectiva.com Sun Apr 7 22:49:57 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Sun, 7 Apr 2002 18:49:57 -0300 Subject: [Python-Dev] Parameters in message headers Message-ID: <20020407184957.A1469@ibook.distro.conectiva> Hi Barry! I've had some problems with messages which doesn't follow the RFC when regarding to parameter building. More specificaly, I've found the following header in a message (in one line): Content-Type: multipart/mixed; boundary = b71528a4d486932a8bdd3c50d1048d7c7 This breaks the email package because, following the RFC, there should be no spaces surrounding the "=" symbol, and the Message class, as expected, breaks this spliting in "=". OTOH, the RFC defines parameters as follows: parameter := attribute "=" value attribute := token ; Matching of attributes ; is ALWAYS case-insensitive. value := token / quoted-string token := 1* So, there's no chance of finding a space surrounding "=", unless the parameter is not written as defined in the RFC. Thus, it's safe to strip the name and value fields. If you choose to keep following the RFC strictly, I'll understand. OTOH, parsing bad written headers safely would be nice as well. I can send a patch, if necessary. Thanks! -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From tim.one@comcast.net Sun Apr 7 23:02:16 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 07 Apr 2002 18:02:16 -0400 Subject: [Python-Dev] Searching Python docs (Was: Python sidebar forMozilla/Netscape) In-Reply-To: <3CAE7EE8.1C3445AA@ActiveState.com> Message-ID: [David "cheerleader" Ascher] > ... > I am convinced that some recent developments such as the better release > management, the filling in of the standard library and documentation, > the cleanup and maturing of the language are all factors which overall > much outweigh the pains that some feel with respect to managing multi- > version code bases or coping with language evolution. > > At the end of the day, Python is still just as fun to program in as it > ever was. > > --David "cheerleader" Ascher What he said, except it's even more fun and more productive for me with every well-considered new gimmick. Sis boom rah, bro, sis boom rah. From guido@python.org Sun Apr 7 23:08:01 2002 From: guido@python.org (Guido van Rossum) Date: Sun, 07 Apr 2002 18:08:01 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Sun, 07 Apr 2002 10:09:42 +0200." <02040710094202.05787@arthur> References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <02040710094202.05787@arthur> Message-ID: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> Alex, please try brevity. :-) > It would help to some degree if the "stable" releases could be shown > to have SOME kind of enhancement. Not as much as if language > changes which did not break existing correct code could also be > there, but some. Modules such as email are a big part of the draw > of 2.2, for example (not quite as big as language-level enhancements, > but big). Now there's a good proposal. But it costs time and effort that takes away from our main mission (and PythonLabs is already under a lot of pressure). I wonder if there's a commercial market for this? Maybe ActiveState could back-port important packages or language changes and insert them into ActivePython? [proposal to semi-backport bool snipped] > Thus it would help us send the message I think prospective users > want to hear -- that Python stands for stability AND change, that > there is not necessarily a contradiction between the two goals. In > a very minor way, to be sure. Given the requirements you give elsewhere in your message, I'm not sure how to satisfy all these constraints. Let's be concrete. Should we backport stuff to 2.1 (the earliest release I'm willing to consider)? If so, what? Look at the "What's New?" section in http://www.python.org/2.2/. The new Windows installer has already been backported to 2.1. Type-class unification is obviously out. So are interators and generators (too much effort). So is // division (I think). Returning longs instead of raising OverflowError on int operations would probably break too much code, so it's better not to backport. That leaves the email package and xmlrpclib (both available as separate distros!), and large file support -- which I believe *did* get ported to 2.1.2. > More relevantly, I'm not sure I had managed to explain exactly why I > think "stability and change" IS important to Python adoption in a > certain category of software shops. Unfortunately, it's not clear what adaptation in that (apparently very change-resistant) category buys *Python*. IOW why should we, mostly volunteers working for the community, care? > What numbering or naming scheme we adopt has importance in > determining what we communicate, but before that, deciding what we > DO want to communicate is probably more important. Indeed. But I'd love it if all we need to do is change our communication patterns, because that takes much less effort than fully maintaining two separate release tracks. > I think such a "retroactive" rebranding of 2.2 as "bleeding-edge" would > be a serious blow to those of us who have accepted it as stable > enough as our base over the last 3 months -- in books, magazine > articles, advocacy within our firms and outside of them, etc. For > example, AB Strakt has decided 2.2 was stable enough for us (and > it was being presented as such) -- our new code now uses > "for x in somedict:", "if x in somedict:", etc, for example -- now > hearing that it wasn't, and 2.2.X micro-releases might introduce > backwards-incompatible changes, would be a real problem indeed. I'm happy to hear that 2.2 is stable for you -- exactly as I intended and expected. But what confuses me is that while you're happy with 2.2's stability, you don't want to testify of this fact to the more conservative shops. Rather than telling them "honestly, I cannot recommend 2.2", why don't you tell them "I've used 2.2 for three months and it's as solid as a rock"? > I perceive Booleans as small (albeit negatively), but both our CTO and > principal investor have vehemently posted against them, so it would > be hard to 'sell' them even if I were quite favorable. If I didn't suspect that you, the CTI and the P.I. together are 75% of the company, I would say that they are micromanaging you. :-) > Still, if that was the one hurdle to jump to get to a "stable and > changing" release, who knows. But if jumping that small hurdle only > gets to more of the same, either unstability or feature-frozen > versions, the carrot may be too small. I suspect that the people who complain that Python's rate of change is too fast are misguided. There's a lot of fuss on the newsgroup about potentially breaking code and undesired changes, and before you know it, everybody just *believes* that Python is unstable and changes too fast. But if you try it, you find very little incompatibility, and what's there is easily dealt with. The people on the newsgroup who complain that the introduction of bool is such a major change that it would only be acceptable in a major release (i.e. 3.0) vastly overestimate its impact, which is minuscule (zero in most cases). Logajan's main gripe (and also McMillan's biggest effort in keeping his codebase cross-version portable) was about things that changed between 1.5.2 and 2.0 -- that *was* a major release. And at the time we didn't have PEPs 4-6, there was no warning module, and so on. So maybe there's only (or mostly) a problem of *perception*? --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@pythoncraft.com Sun Apr 7 23:10:21 2002 From: aahz@pythoncraft.com (Aahz) Date: Sun, 7 Apr 2002 18:10:21 -0400 Subject: [Python-Dev] Historical backups? Message-ID: <20020407221021.GA2403@panix.com> Out of curiosity, is anyone currently dumping CVS to CDROM once a month or so? If not, do people think this would be a Good Idea? (I'm asking because search.python.org got hacked, and it occurs to me that having a long-term baseline might be a Good Idea.) (Let's first get agreement on whether this is a Good Idea, then worry about how to do it, okay?) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From niemeyer@conectiva.com Sun Apr 7 23:54:44 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Sun, 7 Apr 2002 19:54:44 -0300 Subject: [Python-Dev] Re: Parameters in message headers In-Reply-To: <20020407184957.A1469@ibook.distro.conectiva> References: <20020407184957.A1469@ibook.distro.conectiva> Message-ID: <20020407195444.A2199@ibook.distro.conectiva> > I've had some problems with messages which doesn't follow the RFC when > regarding to parameter building. More specificaly, I've found the > following header in a message (in one line): [...] Never mind. I've just read your announce: --- - More liberal acceptance of parameter formatting, e.g. this is now accepted: Content-Type: multipart/mixed; boundary = "FOO" I.e. spaces around the = sign. --- Thank you! -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From barry@zope.com Mon Apr 8 00:27:48 2002 From: barry@zope.com (Barry A. Warsaw) Date: Sun, 7 Apr 2002 19:27:48 -0400 Subject: [Python-Dev] Historical backups? References: <20020407221021.GA2403@panix.com> Message-ID: <15536.54772.901785.884218@anthem.wooz.org> >>>>> "A" == Aahz writes: A> Out of curiosity, is anyone currently dumping CVS to CDROM once A> a month or so? If not, do people think this would be a Good A> Idea? I've got daily backups of the cvs repos dating back to 9-Nov-2001. I'm assuming that XS4ALL is doing proper backups of the directory (~barry/archives on creosote), so I'm not too worried. -Barry From tim.one@comcast.net Mon Apr 8 00:49:51 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 07 Apr 2002 19:49:51 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204070709.g3779aJ32707@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Tim] >> Perhaps you're saying that PLabs should take that over? [Guido] > No, this is for all active developers. How does that differ from PLabs? There are only a few active developers outside of PLabs who aren't "one issue" developers, and I wouldn't consider even *asking* Martin or Michael to take on more than they already do. If a new plan involves docs, it necessarily sucks Fred in; if a new plan involves the PLabs Windows installer, it necessarily sucks me in; and new plan or old, you're hopelessly sucked in . The sum of all active developers isn't enough to tackle anything truly new. > I agree that we don't have to change what we do very much, but I'm > looking for a way to "spin" it to satisfy the more conservative user > community (the ones who still like Python but are worried about the > future). If they write a Python script for 1.5.2, and it doesn't work in Python 58.7.9, they're unhappy. I'm thinking of Logajan, Cave and Rubin there, as they're complaining the loudest these days. They can't be spun into accepting that any backward incompatibility is defensible. I assume almost all of that would go away if incompatible language changes also changed the extensions used for Python files (like, say, .py3 for Python 3). > Since there are different "use cases", this may be hard or impossible, > or we may have to explain it differently for different folks. A possibility I haven't rejected is give up spin and just acknowledge that Python can't be all things to all people here: some people are going to lose. Unfortunate, but that's life if so. > For example, developers who distribute useful Python programs want to > have a way to ensure that code they write gets run on a compatible > Python version. That's not what the people above are saying, though: they want all future Pythons to be compatible with everything they wrote 4 years ago. If a reasonable developer appeared who asked for what you're suggesting, I'd like to try to help them. > We could make it easier to have different X.Y releases installed on the > same system; this is already pretty easy on Unix if you know about "make > altinstall". I have at least 4 versions of Python installed on each of my Windows boxes, but there are limits to how deep we can push that (e.g., a given user can associate .py with at most one interpreter, and before Win2K that was at most one association per machine; *I* don't care, because I never use any Windows gimmick that looks at the .py association; if I did care, I'd be screwed, and because of how Windows works, not because of how Python works). > If we invent a standard way to specify the required version (or range of > versions?) then that might be sufficient. You shot down my #! proposal > for Windows, I didn't see that it addressed a real problem. > but maybe we can do something else. (Obviously this only works for > programs, not for modules or packages. Package authors pretty much > know the drill.) There are lots of ways, but I don't have practical experience with any of them as I'm not change-aversive. Possibilities include specifying minimal version numbers required of the language and/or of specific modules at import sites, relying on a strict backward compatibility guarantee to ensure that any later version will also work OK; and keeping all old versions of all modules around in all future releases, viewing specified version numbers as upper bounds instead of lower bounds. Another possibility is to work in terms of versioned interfaces, and rather than import a module directly, ask the system for any module object that's compatible with a specified version of a specified interface. ... >> I don't think anyone is going to buy that as "maintaining" the 2.1.x >> series; I sure don't. > Maybe we need to use a different word? Barry noted that 2.1.x is the > most popular "stable" release and will probably remain that. So maybe > we should say this is the "recommended" release. And then maybe we > need to revise the decision not to make a Windows installer. Or maybe > we *do* need to spend more effort backporting patches (not necessarily > PLabs, but *somebody*). Who? Anthony Baxter withdrew from the 2.1.x line after 2.1.2 -- he was tired of doing it. He popped up again for 2.1.3 under the assurance that it was going to be a trivial effort. I have a real problem when a group of people demands something but won't volunteer anything to make it happen except their complaints; if they were paying us, fine, but they're not offering funding either. > Another way to keep adding value to older releases is by backporting > the build procedure enhancements. Isn't this dangerous? Build config seems very delicate to me. > Once we've figured out how to build Python 2.3 on, say, some ancient > version of AIX, maybe somebody should port that back to 2.2 and 2.1. If they can swear it's not going to break some other Unix mutant, maybe. > That's not a language change, but still improves the usability of the > release. (For some folks. Maybe the AIX user community is small enough > that we shouldn't care, but there are other platforms with similar > issues. I recall that you backported the new 32 bit Windows installer > to 2.1.2; maybe the recent fixes could be backported to 2.1.3.) I was actually reluctant to check those into 2.2.1: the complaint came from an XP user, and I don't have an XP box to test it on. I believe I fixed a *similar* complaint on my Win2K box, but who knows whether that fixes it for XP too? I don't; I can only guess. Ditto for its effects on ME -- I'm using installer settings I never tried before, and I can't know whether there may be a glitch on ME. It seems to work OK on Win98SE too, so I took a calculated risk and checked it in to 2.2.1. But I won't be shocked if it breaks Windows installation on some version of Windows I don't have. > OK, so I *am* proposing to do something different. Maybe it's not > very attractive to work on old releases. But Zope Corp does it all > the time (and the fact that Zope 2.x requires Python 2.1.x may be an > additional motivator). If you haven't noticed yet , you don't control a fraction of the resource a project of Python's complexity requires. It's harder to maintain old Python releases than it is to maintain old Zope releases, because Python is much closer to the metal: testing on Linux doesn't say much about what will happen on HP-UX or Solaris; testing on Windows doesn't say much about what will happen on Macs; and so on. Testing a x-platform product requires a lot of warm bodies and a lot of time. Without more volunteers, I don't see it happening. That said, I'll be happy to build a Windows installer for 2.1.3. It is, quite literally, the least I could do . From greg@cosc.canterbury.ac.nz Mon Apr 8 01:22:08 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Mon, 08 Apr 2002 12:22:08 +1200 (NZST) Subject: [Python-Dev] ACCEPTED: PEP 285 In-Reply-To: Message-ID: <200204080022.MAA00417@s454.cosc.canterbury.ac.nz> martin@v.loewis.de (Martin v. Loewis) > people who want to see whether something is True > should write 'True is a'. But only if "from __future__ import yoda" is in effect. :-) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Mon Apr 8 02:23:56 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Mon, 08 Apr 2002 13:23:56 +1200 (NZST) Subject: [Python-Dev] Python sidebar for Mozilla/Netscape In-Reply-To: <20020405163236.GA24796@panix.com> Message-ID: <200204080123.NAA00424@s454.cosc.canterbury.ac.nz> Guido: > Read http://www.google.com/terms_of_service.html I don't see how they can claim "by using this service you agree..." etc. when there is no link to this that I can find anywhere on the Google home page or nearby, or even any reason to suspect its existence. This is surely shrink-wrap taken to extreme! Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Mon Apr 8 02:25:55 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Mon, 08 Apr 2002 13:25:55 +1200 (NZST) Subject: [Python-Dev] Python sidebar for Mozilla/Netscape In-Reply-To: <3CADD652.535193C5@lemburg.com> Message-ID: <200204080125.NAA00427@s454.cosc.canterbury.ac.nz> > (urlopen(), on the other hand, would be a no-no.) Surely it would depend on what the program using urlopen did with the result? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Mon Apr 8 02:55:51 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Mon, 08 Apr 2002 13:55:51 +1200 (NZST) Subject: [Python-Dev] Namespaces In-Reply-To: <20020406004705.GA751@panix.com> Message-ID: <200204080155.NAA00432@s454.cosc.canterbury.ac.nz> Aahz : > Rebinding a does not affect the originally bound object > (unless the originally bound object's reference count goes to zero). > > Any ideas about what to call ? Variable. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Mon Apr 8 03:00:55 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Mon, 08 Apr 2002 14:00:55 +1200 (NZST) Subject: [Python-Dev] Namespaces In-Reply-To: <15534.24071.152009.959329@anthem.wooz.org> Message-ID: <200204080200.OAA00436@s454.cosc.canterbury.ac.nz> barry@zope.com (Barry A. Warsaw): > I'd say that a binding operation is that which binds an object to a > name in a particular namespace. But I'd want to prefix the term > "namespace" with an adjective to be precise. An object's attribute > namespace, a module's namespace, etc. What about elements of a list? Calling them a "list namespace" would be rather confusing... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From jafo-python-dev@tummy.com Mon Apr 8 04:16:48 2002 From: jafo-python-dev@tummy.com (Sean Reifschneider) Date: Sun, 7 Apr 2002 21:16:48 -0600 Subject: [Python-Dev] Historical backups? In-Reply-To: <15536.54772.901785.884218@anthem.wooz.org>; from barry@zope.com on Sun, Apr 07, 2002 at 07:27:48PM -0400 References: <20020407221021.GA2403@panix.com> <15536.54772.901785.884218@anthem.wooz.org> Message-ID: <20020407211648.P16962@tummy.com> On Sun, Apr 07, 2002 at 07:27:48PM -0400, Barry A. Warsaw wrote: >I'm assuming that XS4ALL is doing proper backups of the directory >(~barry/archives on creosote), so I'm not too worried. Based on my work doing backup audits for our clients, I'd have to say that unless regular audits are done you probably shouldn't count on them. ;-/ You may want to run a test recovery before being too confident in it... Sean -- Whenever possible, steal code. -- Tom Duff Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From barry@zope.com Mon Apr 8 04:40:58 2002 From: barry@zope.com (Barry A. Warsaw) Date: Sun, 7 Apr 2002 23:40:58 -0400 Subject: [Python-Dev] Historical backups? References: <20020407221021.GA2403@panix.com> <15536.54772.901785.884218@anthem.wooz.org> <20020407211648.P16962@tummy.com> Message-ID: <15537.4426.451759.73033@anthem.wooz.org> >>>>> "SR" == Sean Reifschneider writes: >> I'm assuming that XS4ALL is doing proper backups of the >> directory (~barry/archives on creosote), so I'm not too >> worried. SR> Based on my work doing backup audits for our clients, I'd have SR> to say that unless regular audits are done you probably SR> shouldn't count on them. ;-/ You may want to run a test SR> recovery before being too confident in it... I totally agree (we've been burned this way before too). That's why I cc'd Thomas. :) (We can take this offline -- I doubt the grubby details will be of much interest to python-dev.) -Barry From jafo@tummy.com Mon Apr 8 04:54:10 2002 From: jafo@tummy.com (Sean Reifschneider) Date: Sun, 7 Apr 2002 21:54:10 -0600 Subject: [Python-Dev] Pondering some changes to python.c... In-Reply-To: <15536.29465.372159.281895@anthem.wooz.org>; from barry@zope.com on Sun, Apr 07, 2002 at 12:26:01PM -0400 References: <20020407042038.O16962@tummy.com> <15536.29465.372159.281895@anthem.wooz.org> Message-ID: <20020407215410.Q16962@tummy.com> On Sun, Apr 07, 2002 at 12:26:01PM -0400, Barry A. Warsaw wrote: > AK> You might want to be careful about LD_LIBRARY_PATH -- if the > AK> executable is built for dynamic linking, and it needs a > AK> library that's not in /usr/lib, mightn't changing > AK> LD_LIBRARY_PATH cause it to fail? > >It might indeed, although some *nixes have ways for the sysadmin to >safely extend the default lookup path (i.e. /etc/ld.so.conf and >ldconfig). If you can't clear LD_*, you probably can't safely run it as SUID. That'd be my inclination... Sean -- "Yes on one, no on two." "Is number one nuke Russia, or number two?" -- _Buckaroo_Banzai_ Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From jafo@tummy.com Mon Apr 8 04:55:42 2002 From: jafo@tummy.com (Sean Reifschneider) Date: Sun, 7 Apr 2002 21:55:42 -0600 Subject: [Python-Dev] Pondering some changes to python.c... In-Reply-To: <02d701c1de28$b13938e0$ced241d5@hagrid>; from fredrik@pythonware.com on Sun, Apr 07, 2002 at 01:38:05PM +0200 References: <20020407042038.O16962@tummy.com> <02d701c1de28$b13938e0$ced241d5@hagrid> Message-ID: <20020407215542.R16962@tummy.com> On Sun, Apr 07, 2002 at 01:38:05PM +0200, Fredrik Lundh wrote: >see /Misc/setuid-prog.c Right... I'm thinking that some of that functionality could be built into python.c or a "suidpython.c". Sean -- "Engineering Tablets? Does that mean if I swallow one, I'll be an engineer?" -- Evelyn Mitchell Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From fdrake@acm.org Mon Apr 8 05:37:29 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 8 Apr 2002 00:37:29 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204072119.g37LJiV08575@pcp742651pcs.reston01.va.comcast.net> References: <3CB0178C.28107.5727BDC2@localhost> <3CB075FA.17238.58989C59@localhost> <200204072119.g37LJiV08575@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15537.7817.843061.595855@grendel.zope.com> Guido van Rossum writes: > I think we were too careless with ConfigParser. I think I was on vacation when ConfigParser was added, and I still remember being quite surprised that it landed in the Python CVS. I'd love to see that module frozen (in the sense of "let's not even *try* to fix it), and come up with a more reasonable API that takes a single syntax (.ini file syntax is fine; let's not invent one). It might be worth looking at the GConf API used in Gnome; I'm not sure how it compares. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From barry@zope.com Mon Apr 8 06:35:51 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Apr 2002 01:35:51 -0400 Subject: [Python-Dev] Re: Stability and change References: <3CAEFC4E.2778AC55@python.org> <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15537.11319.576212.846144@anthem.wooz.org> [Barry] > From my own perspective it seems that 2.1.x is viewed as the stable > release family, and each micro release reaffirms its stability. GvR> That's about right. Maybe we should continue to promote GvR> 2.1.x and relegate 2.2 to the bleeding edge? A simple GvR> rearrangement of the website might be sufficient. That might not be a bad idea. 2.2's got some neat stuff in it, but until it's well documented, and really beat upon, I don't think we can rightly call it stable. From my own perspective, while it's gotten pretty hard to continue to support Python 2.0 (nested scopes, warnings, weak refs, the improved build process, and lib fixes), Python 2.1 seems like a good baseline. > That's a good thing. That 2.2.x is viewed as more experimental is > simply caused by the new type/class stuff, and I don't think there's > much marketing you could do to change that perception. Maybe 2.2 > should have been called 3.0 . GvR> Or maybe 2.3 should become 2.2.3. <0.5 wink> I think the new bool type has already prevented that. -Barry From barry@zope.com Mon Apr 8 06:49:29 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Apr 2002 01:49:29 -0400 Subject: [Python-Dev] Re: Stability and change References: <200204070709.g3779aJ32707@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15537.12137.771457.296841@anthem.wooz.org> GvR> Maybe we need to use a different word? Barry noted that GvR> 2.1.x is the most popular "stable" release and will probably GvR> remain that. So maybe we should say this is the GvR> "recommended" release. And then maybe we need to revise the GvR> decision not to make a Windows installer. Or maybe we *do* GvR> need to spend more effort backporting patches (not GvR> necessarily PLabs, but *somebody*). I think those are all good ideas that help promote the idea of a long-lived stable baseline release. GvR> OK, so I *am* proposing to do something different. Maybe GvR> it's not very attractive to work on old releases. But Zope GvR> Corp does it all the time (and the fact that Zope 2.x GvR> requires Python 2.1.x may be an additional motivator). Note that Jim often brings up his opinion that the /only/ way to guarantee Zope Z.Y.Z works on Python X.Y.Z is to distribute Python with Zope. (This harkens back to JWZ's much earlier opinion that Netscape could never ship using dynamic libraries, because you really can't trust anything about the end user's system). We're really touching on much deeper flaws in software development, with no good answers, IMO. we-fear-change-ly y'rs, -Barry From barry@zope.com Mon Apr 8 06:54:49 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Apr 2002 01:54:49 -0400 Subject: [Python-Dev] Re: Stability and change References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <02040710094202.05787@arthur> Message-ID: <15537.12457.59929.194685@anthem.wooz.org> >>>>> "AM" == Alex Martelli writes: AM> It would help to some degree if the "stable" releases could be AM> shown to have SOME kind of enhancement. Not as much as if AM> language changes which did not break existing correct code AM> could also be there, but some. Modules such as email are a AM> big part of the draw of 2.2, for example (not quite as big as AM> language-level enhancements, but big). And of course, I spend a non-trivial amount of effort to maintain a separate email package that is compatible with Python 2.1 and Python 2.2. But I consciously gave up Python 2.0 (or earlier) support because it became obvious to me that I wanted /some/ of the newer features. But that's really difficult, considering it means I have to keep two cvs repositories in sync (and yes, I've got a bunch of changes that need merging back into Python 2.3). -Barry From barry@zope.com Mon Apr 8 07:05:03 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Apr 2002 02:05:03 -0400 Subject: [Python-Dev] Re: Parameters in message headers References: <20020407184957.A1469@ibook.distro.conectiva> <20020407195444.A2199@ibook.distro.conectiva> Message-ID: <15537.13071.671788.778871@anthem.wooz.org> >>>>> "GN" == Gustavo Niemeyer writes: GN> - More liberal acceptance of parameter formatting, GN> e.g. this is now accepted: Content-Type: multipart/mixed; GN> boundary = "FOO" I.e. spaces around the = sign. GN> Thank you! No problem! :) BTW, this was part of email 1.2 which was released 18-Mar-2002. Current cvs is as 2.0 which has some API changes (with the proper DeprecationWarnings ) and will likely be folded into Python 2.3. -Barry From tim.one@comcast.net Mon Apr 8 07:16:19 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 08 Apr 2002 02:16:19 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15537.12137.771457.296841@anthem.wooz.org> Message-ID: [Barry A. Warsaw] > ... > Note that Jim often brings up his opinion that the /only/ way to > guarantee Zope Z.Y.Z works on Python X.Y.Z is to distribute Python > with Zope. And note that we ship Microsoft's C libraries with Python on Windows, to insulate users from x-release incompatibilities in MS's C libraries. Whether it actually helps Python's users varies across Windows flavors, and also across vagaries of the user's installation and configuration settings. But I think it's important to note that in recent MS OSes, they're trying to make it *easier* for apps to force use of the specific DLLs they ship with (although our Python installer isn't yet doing any of the things necessary to exploit these new capabilities). Don't think Linux is immune either. Upgrading gcc or glibc can "suddenly break" programs that used to work fine. God has a sense of humor about this, though: He usually arranges that breakage show up first in obscure libm endcases . > ... > We're really touching on much deeper flaws in software development, > with no good answers, IMO. Indeed so! Python's not the only thing that changes in incompatible ways over time: *everything* does, including the compilers, system libraries, system tools, and OS policies. The first textbook I had on mathematical logic was Stephen Kleene's (yes, that Kleene) cheerfully named "Mathematical Logic". The first chapter contained a warning to the student: It will be very important as we proceed to keep in mind this distinction between the logic we are studying (the object logic) and our use of logic in studying it (the observer's logic). To any student who is not ready to do so, we suggest that he close the book now, and pick some other subject instead, such as acrostics or beekeeping. For this reason, I suggest naming the hypothetical "forward and backward compatible forever more" Python branch the "Python Acrostics and Beekeeping" branch. you-can't-spell-pablum-without-pab-ly y'rs - tim From tim.one@comcast.net Mon Apr 8 07:30:09 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 08 Apr 2002 02:30:09 -0400 Subject: [Python-Dev] GC, flags, and subtyping In-Reply-To: <09ca01c1dc5f$18a16a00$1101a8c0@boostconsulting.com> Message-ID: [David Abrahams] > ... > P.S. One proviso: I find TeX to be cumbersome at best and hair-pullingly > frustrating at worst. If I had to write TeX I probably would just submit > code comprehensibility patches and stop there. I'm probably going to > start with code patches anyway... Don't let LaTeX stand in your way: if you have useful docs to contribute, plain text (and the plainer the better) works fine. Several people can convert to proper Python LaTeX in their sleep, and I suspect that's what Fred actually does in his sleep. I feel very comfortable making this offer, since not one of the people who can do this even when awake. From aleax@aleax.it Mon Apr 8 07:32:26 2002 From: aleax@aleax.it (Alex Martelli) Date: Mon, 8 Apr 2002 08:32:26 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15537.12457.59929.194685@anthem.wooz.org> References: <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> Message-ID: <0204080832260F.05787@arthur> On Monday 08 April 2002 07:54, Barry A. Warsaw wrote: > >>>>> "AM" == Alex Martelli writes: > > AM> It would help to some degree if the "stable" releases could be > AM> shown to have SOME kind of enhancement. Not as much as if > AM> language changes which did not break existing correct code > AM> could also be there, but some. Modules such as email are a > AM> big part of the draw of 2.2, for example (not quite as big as > AM> language-level enhancements, but big). > > And of course, I spend a non-trivial amount of effort to maintain a > separate email package that is compatible with Python 2.1 and Python > 2.2. But I consciously gave up Python 2.0 (or earlier) support > because it became obvious to me that I wanted /some/ of the newer > features. > > But that's really difficult, considering it means I have to keep two > cvs repositories in sync (and yes, I've got a bunch of changes that > need merging back into Python 2.3). How does this amount of effort (keeping two CVS repositories in sync) compare to maintaining just one tree but with "feature-test macros" as one might try to do in C? In other words, how much help would it be if (for example) Python's compiler optimized away tests of the form, e.g., if __version__ > ( 2, 2 ) : as it does now with -O for "if __debug__:" but even more deeply (so even some "syntax errors" in the optimized-away conditional would be OK -- catering for future syntax changes at the add-a-keyword level, e.g.)? Maybe the obfuscated-with-version-tests merged source would be too hard to maintain -- and maybe not, so I'm quite curious about what your experience suggests here. Intuitively, it might seem a version test could perhaps help two or more versions be alive at the same time, but I know of so much controversy over C's #if (mostly, IMHO, because of its over-use) that I wonder. Alex From barry@zope.com Mon Apr 8 07:33:45 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Apr 2002 02:33:45 -0400 Subject: [Python-Dev] Re: Stability and change References: <15537.12137.771457.296841@anthem.wooz.org> Message-ID: <15537.14793.320840.367921@anthem.wooz.org> >>>>> "TP" == Tim Peters writes: TP> Don't think Linux is immune either. Upgrading gcc or glibc TP> can "suddenly break" programs that used to work fine. God has TP> a sense of humor about this, though: He usually arranges that TP> breakage show up first in obscure libm endcases . Or upgrade a supposedly backwards compatible library, like, oh I dunno, libz? >> ... We're really touching on much deeper flaws in software >> development, with no good answers, IMO. TP> Indeed so! Python's not the only thing that changes in TP> incompatible ways over time: *everything* does, including the TP> compilers, system libraries, system tools, and OS policies. Heck, I wash my pillow and I can't sleep right for weeks. TP> For this reason, I suggest naming the hypothetical "forward TP> and backward compatible forever more" Python branch the TP> "Python Acrostics and Beekeeping" branch. TP> you-can't-spell-pablum-without-pab-ly y'rs - tim Or how about the "Garth Releases" (named after character in Wayne's World who utters the immortal truth "We fear change"). -Barry From anthony@interlink.com.au Mon Apr 8 07:35:30 2002 From: anthony@interlink.com.au (Anthony Baxter) Date: Mon, 08 Apr 2002 16:35:30 +1000 Subject: [Python-Dev] 2.1.3 release process is underway... Message-ID: <200204080635.g386ZUj06058@mbuna.arbhome.com.au> As those who are on the checkins list will have noticed, I'm working through PEP 102 for the 2.1.3 release at the moment... Shouldn't affect anyone, but just so you know... Anthony From skip@pobox.com Mon Apr 8 07:45:22 2002 From: skip@pobox.com (Skip Montanaro) Date: Mon, 8 Apr 2002 01:45:22 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15537.11319.576212.846144@anthem.wooz.org> References: <3CAEFC4E.2778AC55@python.org> <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <15537.11319.576212.846144@anthem.wooz.org> Message-ID: <15537.15490.631372.551394@12-248-41-177.client.attbi.com> BAW> [Barry] >> From my own perspective it seems that 2.1.x is viewed as the stable >> release family, and each micro release reaffirms its stability. GvR> That's about right. Maybe we should continue to promote GvR> 2.1.x and relegate 2.2 to the bleeding edge? A simple GvR> rearrangement of the website might be sufficient. BAW> That might not be a bad idea. 2.2's got some neat stuff in it, but BAW> until it's well documented, and really beat upon, I don't think we BAW> can rightly call it stable. This sounds more or less like the Linux kernel, except the meaning of the odd and even minor revisions are reversed. (Guido: You mentioned before that you have trouble keeping the Linux stable/experimental kernel numbering straight. Just execute "uname -a" and look at the kernel's minor version number. I can almost guarantee you aren't running an experimental kernel. If so, you don't have enough work to do. ;-) Personally, I think the way it's done in the Linux world is fine and other than the initial turmoil caused by switching between the current release scheme and a Linux-like release scheme I think it would probably play well in Peoria (that is, c.l.py). You'd dispense with alphas and betas as they are currently done and use the micro releases for that. Let's assume even-numbered minor releases are experimental, and odd-numbered ones are stable. The early micro releases of experimental versions would be where most of the turmoil takes place. New stuff is added, and maybe ripped back out if it doesn't work. As the micro release numbers increase, the amount of turmoil reduces, the feature set is frozen, and eventually 2.2.N becomes both 2.3.0 and 2.4.0. The 2.3.x branch gets essentially nothing but bug fixes and new development then pours into 2.4.x. The advantage over the current scheme in my mind is that we have a lot of turmoil in the release train near to a release. I think the ideal scenario would be the exact opposite. Today, little activity takes place during the time between one release and the first alpha of the next release. After 2.x.alpha1 is announced, the repository churns a lot. The closer and closer you get to the first beta, the more agitated things seem to become. I think this is because you try to set and adhere to specific release dates and because each release is meant to be "stable". What's Fred Brooks' aphorism about software? "Plan to throw one away." That can be every other set of minor releases. GvR> Or maybe 2.3 should become 2.2.3. <0.5 wink> BAW> I think the new bool type has already prevented that. Why? If you postulate that 2.even.x become the experimental release branches, then 2.2.3 with a bool type makes perfect sense. Once the bool type is in and you're satisfied that you've accounted for most of the necessary changes, you make a micro release. No big deal. Create tgz and zip files, maybe a Windows installer. On to the next one. release-early-release-often-ly, y'rs, Skip From Anthony Baxter Mon Apr 8 07:52:11 2002 From: Anthony Baxter (Anthony Baxter) Date: Mon, 08 Apr 2002 16:52:11 +1000 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Message from Skip Montanaro of "Mon, 08 Apr 2002 01:45:22 EST." <15537.15490.631372.551394@12-248-41-177.client.attbi.com> Message-ID: <200204080652.g386qBS06363@mbuna.arbhome.com.au> >>> Skip Montanaro wrote > GvR> Or maybe 2.3 should become 2.2.3. <0.5 wink> > BAW> I think the new bool type has already prevented that. > > Why? If you postulate that 2.even.x become the experimental release > branches, then 2.2.3 with a bool type makes perfect sense. Once the bool > type is in and you're satisfied that you've accounted for most of the > necessary changes, you make a micro release. No big deal. Create tgz and > zip files, maybe a Windows installer. On to the next one. Hm. I'd like to keep the "stable between 2.x.a and 2.x.b" thing going, if only because the lib/python-x.y directory is shared between releases. Anthony From Anthony Baxter Mon Apr 8 07:53:35 2002 From: Anthony Baxter (Anthony Baxter) Date: Mon, 08 Apr 2002 16:53:35 +1000 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Message from barry@zope.com (Barry A. Warsaw) of "Mon, 08 Apr 2002 02:33:45 -0400." <15537.14793.320840.367921@anthem.wooz.org> Message-ID: <200204080653.g386rZm06438@mbuna.arbhome.com.au> >>> Barry A. Warsaw wrote > Or how about the "Garth Releases" (named after character in Wayne's > World who utters the immortal truth "We fear change"). Excellent! I think this is a great name for 2.1.3 :) Anthony From barry@zope.com Mon Apr 8 08:15:10 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Apr 2002 03:15:10 -0400 Subject: [Python-Dev] Re: Stability and change References: <3CAEFC4E.2778AC55@python.org> <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <15537.11319.576212.846144@anthem.wooz.org> <15537.15490.631372.551394@12-248-41-177.client.attbi.com> Message-ID: <15537.17278.333109.580542@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: GvR> Or maybe 2.3 should become 2.2.3. <0.5 wink> BAW> I think the new bool type has already prevented that. SM> Why? If you postulate that 2.even.x become the experimental SM> release branches, then 2.2.3 with a bool type makes perfect SM> sense. Given that interpretation, I'd agree. -Barry From barry@zope.com Mon Apr 8 08:20:09 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Apr 2002 03:20:09 -0400 Subject: [Python-Dev] Re: Stability and change References: <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> <0204080832260F.05787@arthur> Message-ID: <15537.17577.734317.166647@anthem.wooz.org> >>>>> "AM" == Alex Martelli writes: AM> How does this amount of effort (keeping two CVS repositories AM> in sync) compare to maintaining just one tree but with AM> "feature-test macros" as one might try to do in C? In other AM> words, how much help would it be if (for example) Python's AM> compiler optimized away tests of the form, e.g., if AM> __version__ > ( 2, 2 ) : as it does now with -O for "if AM> __debug__:" but even more deeply (so even some "syntax errors" AM> in the optimized-away conditional would be OK -- catering for AM> future syntax changes at the add-a-keyword level, e.g.)? AM> Maybe the obfuscated-with-version-tests merged source would be AM> too hard to maintain -- and maybe not, so I'm quite curious AM> about what your experience suggests here. Intuitively, it AM> might seem a version test could perhaps help two or more AM> versions be alive at the same time, but I know of so much AM> controversy over C's #if (mostly, IMHO, because of its AM> over-use) that I wonder. Part of the problem is implementation details (i.e. avoiding the use of Python 2.2 features in standalone, but not in the Python cvs tree). Part of it is due to the nature of distutils package layout (i.e. you need to put setup.py, README, etc. someplace, and distutils seems to strongly discourage it being in th epackage). Part of it's also that I feel I have more leeway to change things in the standalone package. But at some point it will stop making sense to distribute a separate package. -Barry From xscottg@yahoo.com Mon Apr 8 08:52:43 2002 From: xscottg@yahoo.com (Scott Gilbert) Date: Mon, 8 Apr 2002 00:52:43 -0700 (PDT) Subject: [Python-Dev] Array Enhancements In-Reply-To: <20020405203029.19286.qmail@web12903.mail.yahoo.com> Message-ID: <20020408075243.79423.qmail@web12905.mail.yahoo.com> Thanks for the various replies. As suggested by a few, I'll take this to the Numarray folks and see where it goes from there. Just to respond to a few of the points though... I've put all my responses in one message to wrap things up. Tim Peters wrote: > > Sounds like a PEP to me. > My initial response to reading this was a loud "ugh" as I envisioned red tape swarming around for what I would consider to be a pretty simple patch. I mean, I just wanted to hack in some new typecodes... After thinking about things for a while though, I've come to the conclusion that the builtin Python array module does need a real reworking. Even though one ships with the standard baseline, it's getting reinvented again and again. I hope the Numarray guys are doing a bang up job with their NDArray type (I've looked at it briefly, but I don't really understand it yet...). I suspect that most of the ufuncs and other stuff those guys are doing are too special purpose to be part of the standard Python baseline, but I would very much like to see a single usable array type become the standard. I'd be willing to do PEP grunt work for that. Tim Peters also wrote: > > ... > > *** I really need complex types. And more than the functionality > > provided by Numeric/Numarray, I need complex integer types. > > This will meet resistance, as it's a pile of code of no conceivable > use to the vast majority of Python users. That is, "code bloat". > Instead the array type should be subclassable, and extreme > special-purpose hair like "complex integers" should be supplied by > extension modules. It's not that much bloat. It would be a setitem and getitem pair for each new type. I'll give you that most people don't need "fixed point complex arrays". Guido van Rossum wrote: > You'll have to consider: is it important to be able to read pickled > arrays on previous Python releases, or it that not a requirement? If > it's not, you should probably add a new pickle code for pickled > arrays, and do an implementation that writes; Nope, we ship the version of Python we want them to use with our applications. Did you guys really make it possible to unpickle a Unicode string in versions of Python that were pre Unicode? I would think new features should only work in new versions... Guido also wrote: > > Ehm, 'u' is already taken (Unicode). > That must have snuck in there sometime after 2.2 I guess. Guido also wrote: > > *** The ability to construct an array object from an existing C > > pointer. We get our memory in all kinds of ways (valloc for page > > aligned DMA transfers, shmem etc...), and it would be nice not to > > copy in and copy out in some cases. > > But then you get into ownership issues. Who owns that memory? Who > can free it? What if someone calls a method on the array that > requires the memory to be resized? > > But it's a useful thing to be able to do, I agree, and it shouldn't be > too hard to add a flag that says "I don't own this memory" -- which > would mean that the buffer can't be resized at all. I pictured this working like CObjects do where you pass in a destructor for when the reference count goes to zero. Possibly also passing in a realloc function. If the realloc function is null, then an exception is raised when someone tries to resize the array. This means there would need to be a C visible API for building array objects around special types of memory though. Guido also wrote: > Since arrays are all about compromises that trade flexibility for > speed and memory footprint, you can't have a one size fits all. :-) Bahh. I don't think getting a good general purpose Python object that represents arbitrary C arrays is all that impossible. C arrays just don't do that much. Besides I didn't say "one size fits all", I said "one size fits all my needs". That "my" is important (at least to me :-) Guido also wrote: > > Well if someone authoritative tells me that all of the above is a > > great idea, I'll start working on a patch and scratch my plans to > > create a "not in house" xarray module. > > It all depends on the quality of the patch. By the time you're done > you may have completely rewritten the array module, and then the > question is, wouldn't your own xarray module have been quicker to > implement, because it doesn't need to preserve backwards > compatibility? Yup, I think I would be done with my xarray module by now if I had written it instead of taking this route. It would also have the disadvantage that it doesn't play nice with anyone else. I now think the best bet is to replace the array module with something flexible enough to: 1) do what it currently does 2) do what the Numarray guys need 3) do what I need Guido also wrote: > An alternative might be a separate bit-array implementation: it seems > that the bit-array won't share much code with the regular array (of > any flavor), so why not make it a separate type? Yup. It would be nice if a bitarray was actually the same type, but having code like: if (o->is_bitarray) { /* do something */ } else { /* do every other byte addressable type */ } is a little ugly. David Ascher wrote: > > I just realized that multi-dimensional __getitem__ shouldn't be a > > big deal. The question is, given the above declaration, what a[0] > > should return: the same as a[0, 0] or a copy of a[0, 0:20000] or > > a reference to a[0, 0:20000]. > > Or a ValueError? In the face of ambiguity, refuse the temptation to > guess. > Yup. I think there should be a base array type that raises a ValueError or similar, and derived array types can implement slice references or slice copies as need be. David also wrote: > > Why does submitting a patch to arraymodule seem an easier path than > modifying numarray or numpy to support what's needed? I believe that > the goals of numarray aren't that different from what Scott is trying > to do (memory management APIs, etc.). > Well, part of my preference for modifying arraymodule.c instead of Numarray is that I very quickly understood what's going on in arraymodule.c, and a patch is pretty obvious. Looking at Numarray, I just don't get it yet. Please take this as a shortcoming in my abilities. Numarray does appear to be the heir-apparent though, so I'll give it a better look. I also assumed that the Numarray folks would play nice with the standard array module. So if I could get what I wanted out of array, then I could leverage Numarray when the opportunity arose. David also wrote: > > I'd like to see fewer multi-dimensional array objects, not more... > I agree completely. In fact, I'd like to see one official one distributed with the baseline. Perry Greenfield wrote: > [ a whole bunch of interesting things ] I think I'll try to bring those up on the Numarray list. Cheers, -Scott Gilbert __________________________________________________ Do You Yahoo!? Yahoo! Tax Center - online filing with TurboTax http://taxes.yahoo.com/ From aleax@aleax.it Mon Apr 8 09:20:04 2002 From: aleax@aleax.it (Alex Martelli) Date: Mon, 8 Apr 2002 10:20:04 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> References: <02040710094202.05787@arthur> <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> Message-ID: <0204081020040G.05787@arthur> On Monday 08 April 2002 00:08, Guido van Rossum wrote: > Alex, please try brevity. :-) > > > It would help to some degree if the "stable" releases could be shown > > to have SOME kind of enhancement. Not as much as if language > > changes which did not break existing correct code could also be > > there, but some. Modules such as email are a big part of the draw > > of 2.2, for example (not quite as big as language-level enhancements, > > but big). > > Now there's a good proposal. But it costs time and effort that takes > away from our main mission (and PythonLabs is already under a lot of > pressure). You know PythonLabs's missions and goals better than I do. If such goals weigh "making a new and better Python" far more highly than helping spread and preserve the use of Python, then you may well be right that the extra effort for (e.g.) making email part of the standard distribution vs the current "maintaining a separate distro" would be a bad allocation of scarce resources. > I wonder if there's a commercial market for this? Maybe > ActiveState could back-port important packages or language changes and > insert them into ActivePython? If I understand correctly, ActiveState has backed off (because of not sufficient revenue returns) from "owning" Hammond's win32all extensions, which are truly indispensable to firms using Python for Windows-only development. If win32all did not "have a commercial market" (sufficient to justify ActiveState's investments), then I don't see what other packages could. > [proposal to semi-backport bool snipped] > > > Thus it would help us send the message I think prospective users > > want to hear -- that Python stands for stability AND change, that > > there is not necessarily a contradiction between the two goals. In > > a very minor way, to be sure. > > Given the requirements you give elsewhere in your message, I'm not > sure how to satisfy all these constraints. Let's be concrete. Should > we backport stuff to 2.1 (the earliest release I'm willing to > consider)? If so, what? Look at the "What's New?" section in > http://www.python.org/2.2/. The new Windows installer has already > been backported to 2.1. Type-class unification is obviously out. So > are interators and generators (too much effort). So is // division (I > think). Returning longs instead of raising OverflowError on int > operations would probably break too much code, so it's better not to Something requiring much effort is not necessarily "obviously out": it depends on how important you judge the results of those effort versus the results you could have by spending the effort elsewhere. In terms of "how it looks to actual and prospective users of Python" rather than "how much effort would it be to implement", iterators (not generators) and // division look good -- enabling "for x in dict" and "if x in dict" are two features of instant appeal -- language enhancements that do not break previously correct code. It's of course possible to construct a program that would break due to any such change, because it expects and relies upon a specific exception being raised when such usage is attempted -- when the usage becomes legal and meaningful, crack. But in most cases that would be a "sea-lawyer" level issue. You're probably right that, alas, overflow on int operations wouldn't be feasible to remove (I don't know). Syntax changes that make some source code legal where it didn't use to be, e.g. // , are probably at the other extreme -- in practice one would not rely on e.g. compile('a//b') throwing a SyntaxException and thus break if it doesn't. To "be concrete" as you ask, I think that targeting 2.1 as the stable release would not be the best allocation of effort, even though I see others disagree. Basically, the argument against that would be that 2.2 was _announced_ as the new stable release, not as a bleeding edge experimental one, and making it "retroactively experimental" would send exactly the wrong message. So, I think effort would be better spent ensuring that enhancements for 2.3 are well divided between [a] those which are safe to have in some 2.2.* as well, vs [b] those which ARE experimental (break previously correct code) and thus should only go into 2.3. > backport. That leaves the email package and xmlrpclib (both available > as separate distros!), and large file support -- which I believe *did* > get ported to 2.1.2. The fact that email and xmlrpclib are already installable as separate distros on 2.1.* suggests the amount of effort to put them into the standard library of 2.1.something might be modest; therefore, even if the advantage is also modest, it might still be worth doing. > > More relevantly, I'm not sure I had managed to explain exactly why I > > think "stability and change" IS important to Python adoption in a > > certain category of software shops. > > Unfortunately, it's not clear what adaptation in that (apparently very > change-resistant) category buys *Python*. IOW why should we, mostly > volunteers working for the community, care? Assuming one cares about Python being widely used, "capturing" a new "customer" that is highly change-resistant has better returns than getting one who's very change-prone. Exactly because of that user's inertia, they'll stick with Python once they've decided for it -- some other shop who'll gladly have a carefree fling with Python is just as liable to leave it for Ruby, or whatever, tomorrow. Apparently we DO want to promote the spread of Python (otherwise the purpose of the python marketing list and the PSF becomes murky to me). We believe the world will be a better place if more development uses Python rather than different languages. That goal is well served by helping Python acceptance by development shops that do care about stability, because such shops are a substantial fraction of our industry. Over the decades, the people who are now the technical managers and senior developers in such shops have seen too much of their work laid waste by incompatible changes in platforms, and have thus become quite sensitive to that specific technological risk; they've seen some other fraction of their efforts similarly wasted by having been spent on platforms who became "really stable" (i.e., dead) and are thus also sensitized to that antisymmetric risk. These are survival traits in the software jungle. Our industry suffers from so many "bandwagons" (in platforms, methodologies, tools, etc) that learning when NOT to jump on one -- developing a nose for the symptoms of the two opposed risks, excessive/churning rate of change and the "sunset boulevard" syndrome of a slowly dying technology -- make a shop's survival more likely. So, I think we need to avoid triggering either reflex (in sensible cases: we can't really make Logajan happy, I think). That is, IF we want to promote Python's spread -- if we do believe we'll be happier if Python is widely accepted rather than less used than it deserves to be. I'm reasonably sure of that, myself. The more Python is widespread, the more of a "Python market" commercial ventures perceive as existing, the better my chances to find a good Python port to some weird new platform, the better some other enthusiast's chances to find jobs based on Python (or mine, should my current arrangements fail or terminate), the better tools I'm likely to have my pick amongst, and so forth. I didn't think this needed to be re-examined on this thread, but I'm quite game if it does. > > What numbering or naming scheme we adopt has importance in > > determining what we communicate, but before that, deciding what we > > DO want to communicate is probably more important. > > Indeed. But I'd love it if all we need to do is change our > communication patterns, because that takes much less effort than fully > maintaining two separate release tracks. I understand that doing nothing except changing communication patterns entails less effort than other changes. Exactly because of that, the other changes send a stronger message: they bespeak real commitment to "stability AND change" _because_ they don't come for free. Economy Nobel prizes last year all went to economists whose major work has been on this subject: asymmetric-information situations and message-sending. For example, that's how advertising, sponsoring sports, etc, mostly work: a firm sends a message "we really believe in this product" by splurging a lot of money in promoting it, _because_ it's substantial effort -- effort it would make no sense for them to spend if they (who have more info on the product than the prospective buyer) really knew it was a dud. I'm summarizing and oversimplifying, but that's the gist. > > I think such a "retroactive" rebranding of 2.2 as "bleeding-edge" would > > be a serious blow to those of us who have accepted it as stable > > enough as our base over the last 3 months -- in books, magazine > > articles, advocacy within our firms and outside of them, etc. For > > example, AB Strakt has decided 2.2 was stable enough for us (and > > it was being presented as such) -- our new code now uses > > "for x in somedict:", "if x in somedict:", etc, for example -- now > > hearing that it wasn't, and 2.2.X micro-releases might introduce > > backwards-incompatible changes, would be a real problem indeed. > > I'm happy to hear that 2.2 is stable for you -- exactly as I > intended and expected. But what confuses me is that while you're > happy with 2.2's stability, you don't want to testify of this fact to > the more conservative shops. Rather than telling them "honestly, I > cannot recommend 2.2", why don't you tell them "I've used 2.2 for > three months and it's as solid as a rock"? I _do_ testify that all new code I've written or ported to 2.2 has run without a hitch so far. That's quite different from saying that existing code written for 2.1 ran unchanged with 2.2, because it didn't -- nor do I think it reasonably COULD have, mind you. E.g., any time I introspected an instance via dir() I only used to get per-instance attributes -- now I get attributes it gets from its class too; it's an enhancement, but it does fool any such introspection that I had. Unit-tests might help, but unfortunately I relied on doctest for those, so I get a zillion "breakage" reports that aren't all that helpful. Guess I misused doctest (but it's SO much handier than unittest that it's an "attractive nuisance", consarn it!-). Once you do manage to identify the real breakage points, it's not (generally) all that hard to turn 2.1 code into 2.2 code. People who maintain both 2.1 and 2.2 versions of their packages are no doubt better placed to evaluate that. But the message that "only little broke, and fixing it was not too terrible" is not strong enough for the shop looking for stability -- which I guess help explains 2.1's relative longevity, no? > > I perceive Booleans as small (albeit negatively), but both our CTO and > > principal investor have vehemently posted against them, so it would > > be hard to 'sell' them even if I were quite favorable. > > If I didn't suspect that you, the CTI and the P.I. together are 75% of > the company, I would say that they are micromanaging you. :-) No: there's 12 of us, but we also have a strong commitment to *no "code ownership"*. We need to agree on coding standards that are strong enough that we _can't tell_ who wrote what piece -- at least, that's the ideal (and part of XP). And the standards must get consensus, which is no picnic (but the alternatives are worse). > > Still, if that was the one hurdle to jump to get to a "stable and > > changing" release, who knows. But if jumping that small hurdle only > > gets to more of the same, either unstability or feature-frozen > > versions, the carrot may be too small. > > I suspect that the people who complain that Python's rate of change is > too fast are misguided. There's a lot of fuss on the newsgroup about > potentially breaking code and undesired changes, and before you know > it, everybody just *believes* that Python is unstable and changes too > fast. But if you try it, you find very little incompatibility, and > what's there is easily dealt with. The people on the newsgroup who > complain that the introduction of bool is such a major change that it > would only be acceptable in a major release (i.e. 3.0) vastly > overestimate its impact, which is minuscule (zero in most cases). The impact is small (that's my evaluation) in terms of maintenance of existing code (not necessarily all that small in teaching the language or writing about it, but that's another issue). Total breakage of doctests is probably the worst part (maybe it's just what we needed to stop using doctests and go for other approaches instead). Still, "zero in most cases" is NOT the situation with doctest usage today: I do believe I was not the only one to fall into its attractive-nuisance trap. > Logajan's main gripe (and also McMillan's biggest effort in keeping > his codebase cross-version portable) was about things that changed > between 1.5.2 and 2.0 -- that *was* a major release. And at the time > we didn't have PEPs 4-6, there was no warning module, and so on. > > So maybe there's only (or mostly) a problem of *perception*? Mostly, yes, and I do agree that Logajan's impossible to satisfy -- I see no amount of effort that could have kept him happy, given his expressed complaints. But if you're implying that said perception (by prospective users) is unfounded, I disagree. It has a solid basis in reality. Some previously correct code does break at each minor release (doctests first -- as apparently is their "canary in a coal mine" intended role -- but that is quite visible and contributes highly to perception). To some extent ("sea-lawyering" level) that may be inevitable. Moreover, the "status" perceived for releases such as 2.1.2, etc, seems correct to me: you do NOT want to invest substantial effort on them -- isn't this what you're saying throughout this? So, the perception "not quite dead but close, BDFL doesn't care much at all about them" is correct, no? It may be possible to change perception without changing reality, but I have some doubt. "You can fool some of the people all of the time", etc, etc:-). Alex From mal@lemburg.com Mon Apr 8 09:33:21 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 08 Apr 2002 10:33:21 +0200 Subject: [Python-Dev] Pondering some changes to python.c... References: <20020407042038.O16962@tummy.com> Message-ID: <3CB155D1.422AD8C3@lemburg.com> Sean Reifschneider wrote: > > I've been pondering some changes to the main() interpreter code for a > while. I'd like to get some feedback on them. > > One is making changes to the process title to remove the interpreter name. > I've implemented some code to do this, which I'll include below. I realize > that this code won't work for every platform, but for some it would be > quite a nice change. In particular, it would allow things like "killall" > to be able to work on python programs. > > The basics are that I check to make sure all the argv elements are > sequential, then just do "memcpy(argv[0], argv[1], appropriateSize)", and > pad strlen(orig_argv[0]) at the end with '\0'. This is after saving a copy > of the original argv to hand off to Python itself. -1 on making it the default -- there are programs relying on this. +1 on providing a sys module interface for this so that programs can choose the name on their own. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From thomas@xs4all.net Mon Apr 8 09:33:57 2002 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 8 Apr 2002 10:33:57 +0200 Subject: [Python-Dev] Historical backups? In-Reply-To: <20020407211648.P16962@tummy.com> References: <20020407221021.GA2403@panix.com> <15536.54772.901785.884218@anthem.wooz.org> <20020407211648.P16962@tummy.com> Message-ID: <20020408083356.GX16278@xs4all.nl> On Sun, Apr 07, 2002 at 09:16:48PM -0600, Sean Reifschneider wrote: > On Sun, Apr 07, 2002 at 07:27:48PM -0400, Barry A. Warsaw wrote: > >I'm assuming that XS4ALL is doing proper backups of the directory > >(~barry/archives on creosote), so I'm not too worried. > Based on my work doing backup audits for our clients, I'd have to say that > unless regular audits are done you probably shouldn't count on them. ;-/ > You may want to run a test recovery before being too confident in it... We do regular audits. The backups don't go back very far (a week at most) but they do work. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From mwh@python.net Mon Apr 8 11:10:15 2002 From: mwh@python.net (Michael Hudson) Date: 08 Apr 2002 11:10:15 +0100 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape In-Reply-To: Skip Montanaro's message of "Fri, 5 Apr 2002 10:33:58 -0600" References: <3CAD812C.67F70780@lemburg.com> <200204051625.g35GPib13488@pcp742651pcs.reston01.va.comcast.net> <15533.53750.930189.744291@beluga.mojam.com> Message-ID: <2madse5wmw.fsf@starship.python.net> Skip Montanaro writes: > Also, if you notice their links, they redirect through Google > itself: > > http://www.google.com/url?sa=U&start=1&q=http://www.python.org/&e=921 > > (I don't think they used to do this, but I'm not certain.) They did way back when it was google.stanford.edu, then they stopped, and now they've started again. No idea why, of course. Cheers, M. -- ROOSTA: Ever since you arrived on this planet last night you've been going round telling people that you're Zaphod Beeblebrox, but that they're not to tell anyone else. -- The Hitch-Hikers Guide to the Galaxy, Episode 7 From mwh@python.net Mon Apr 8 12:53:03 2002 From: mwh@python.net (Michael Hudson) Date: 08 Apr 2002 12:53:03 +0100 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Tim Peters's message of "Sun, 07 Apr 2002 19:49:51 -0400" References: Message-ID: <2m7kni5rvk.fsf@starship.python.net> I'm only scanning this huge thread, so I may be out of context/repetitive. Tim Peters writes: > [Tim] > >> Perhaps you're saying that PLabs should take that over? > > [Guido] > > No, this is for all active developers. > > How does that differ from PLabs? There are only a few active developers > outside of PLabs who aren't "one issue" developers, and I wouldn't consider > even *asking* Martin or Michael to take on more than they already do. Thank you :) > If a new plan involves docs, it necessarily sucks Fred in; if a new > plan involves the PLabs Windows installer, it necessarily sucks me > in; and new plan or old, you're hopelessly sucked in . The > sum of all active developers isn't enough to tackle anything truly > new. [...] > >> I don't think anyone is going to buy that as "maintaining" the 2.1.x > >> series; I sure don't. > > > Maybe we need to use a different word? Barry noted that 2.1.x is the > > most popular "stable" release and will probably remain that. So maybe > > we should say this is the "recommended" release. And then maybe we > > need to revise the decision not to make a Windows installer. Or maybe > > we *do* need to spend more effort backporting patches (not necessarily > > PLabs, but *somebody*). > > Who? Anthony Baxter withdrew from the 2.1.x line after 2.1.2 -- he was > tired of doing it. I'm reasonbly likely to disappear for a bit after 2.2.1 is done. Doing a maintence release is a lot of work, and it's dull, tedious work (but not thankless!). I think 2.2.1 will be a pretty good release, and if so this will be in large part due to a surge of enthusiasm on my part in the days around the 2.2.1c1 release[1]. This isn't going to happen again for a while -- I want to do some fun hacking (eg. porting psyco to my new iBook), and some maths. I'm reasonably willing to do 222 so long as it's not for a few months. [1] It's also very noticable that I got *far* more feedback about 2.2.1c1 than about 2.2.1c2. Like ten times the amount. > He popped up again for 2.1.3 under the assurance that it was going > to be a trivial effort. I have a real problem when a group of > people demands something but won't volunteer anything to make it > happen except their complaints; Indeed; they can fuck off, in my book. > > Another way to keep adding value to older releases is by backporting > > the build procedure enhancements. > > Isn't this dangerous? Build config seems very delicate to me. Bloody hell yes. > > Once we've figured out how to build Python 2.3 on, say, some ancient > > version of AIX, maybe somebody should port that back to 2.2 and 2.1. > > If they can swear it's not going to break some other Unix mutant, maybe. This is a real issue. There are build fixes for AIX 3 on sf for 221; they're not going in because I'm scared that they'll break something else. There's hope in the work Martin's doing with autoconf. I'd like to see this stuff in 222, if it works out. Cheers, M. -- . <- the point your article -> . |------------------------- a long way ------------------------| -- Cristophe Rhodes, ucam.chat From mwh@python.net Mon Apr 8 12:56:47 2002 From: mwh@python.net (Michael Hudson) Date: 08 Apr 2002 12:56:47 +0100 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Tim Peters's message of "Mon, 08 Apr 2002 02:16:19 -0400" References: Message-ID: <2m4rim5rpc.fsf@starship.python.net> Tim Peters writes: > The first textbook I had on mathematical logic was Stephen Kleene's > (yes, that Kleene) cheerfully named "Mathematical Logic". The first > chapter contained a warning to the student: > > It will be very important as we proceed to keep in mind this > distinction between the logic we are studying (the object logic) > and our use of logic in studying it (the observer's logic). To > any student who is not ready to do so, we suggest that he close > the book now, and pick some other subject instead, such as > acrostics or beekeeping. Can you say "downward Skolem-Loewenheim"? > For this reason, I suggest naming the hypothetical "forward and backward > compatible forever more" Python branch the "Python Acrostics and Beekeeping" > branch. Suits me. Cheers, M. -- I never realized it before, but having looked that over I'm certain I'd rather have my eyes burned out by zombies with flaming dung sticks than work on a conscientious Unicode regex engine. -- Tim Peters, 3 Dec 1998 From mwh@python.net Mon Apr 8 12:57:55 2002 From: mwh@python.net (Michael Hudson) Date: 08 Apr 2002 12:57:55 +0100 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Neil Schemenauer's message of "Sat, 6 Apr 2002 18:07:25 -0800" References: <3CAEFC4E.2778AC55@python.org> <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <20020406180725.A14030@glacier.arctrix.com> Message-ID: <2m1ydq5rng.fsf@starship.python.net> Neil Schemenauer writes: > I think the discussion should focus on whether more effort should be > spent on micro releases. Perhaps 2.2.1 should have been finished before > work was started on 2.3. Mandating that this should happen wouldn't make it happen. > Personally, I don't think so. Me too. Cheers, M. -- If a train station is a place where a train stops, what's a workstation? -- unknown (to me, at least) From mal@lemburg.com Mon Apr 8 13:06:45 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 08 Apr 2002 14:06:45 +0200 Subject: [Python-Dev] checkins list seems to bo down... Message-ID: <3CB187D5.AABC4B1B@lemburg.com> Subject says it all. Any idea ? -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From Anthony Baxter Mon Apr 8 13:25:23 2002 From: Anthony Baxter (Anthony Baxter) Date: Mon, 08 Apr 2002 22:25:23 +1000 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Message from Michael Hudson of "08 Apr 2002 12:53:03 +0100." <2m7kni5rvk.fsf@starship.python.net> Message-ID: <200204081225.g38CPNn10186@mbuna.arbhome.com.au> >>> Michael Hudson wrote > I'm only scanning this huge thread, so I may be out of context/repetitive. You're not the only one :) > > Who? Anthony Baxter withdrew from the 2.1.x line after 2.1.2 -- he was > > tired of doing it. > I'm reasonbly likely to disappear for a bit after 2.2.1 is done. > Doing a maintence release is a lot of work, and it's dull, tedious > work (but not thankless!). I can probably step back in for a bit while Michael steps back. After 2.1.2, the conference and then moving house, I didn't have the time (or the net connection) to keep this going. Michael's done an amazing job on 2.2.1, and I'm so glad he stepped in. However, it's pretty clear to me that trying to keep doing the patch monkey role for multiple releases in a row is a great way to burn out completely. > I'm reasonably willing to do 222 so long as it's not for a few months. The key thing, I think, is to keep on top of the backporting. If it slips, it's an absolute monster to catch up. > > I have a real problem when a group of > > people demands something but won't volunteer anything to make it > > happen except their complaints; > Indeed; they can fuck off, in my book. Absolutely. I believe the phrase "Go Stick Your Head In A Pig" is also appropriate. > > Isn't this dangerous? Build config seems very delicate to me. > Bloody hell yes. me too. I know I try to build releases on everything I can get hold of (the cf.sf.net machines, the dec^Wdigital^Wcompaq^Wwhoever testdrive boxes), but that's a whole lot of work. And that's not including the, ahem, less unixish Unixes like AIX and HP/UX. Unless it's a platform to which I have a lot of exposure (and in the last few years that's been mostly linux and solaris), I'm not going to try and make this stuff work. > This is a real issue. There are build fixes for AIX 3 on sf for 221; > they're not going in because I'm scared that they'll break something > else. *nod* Yep. The chances of a build fix going in that's not completely and utterly clearly the Right Thing to do is almost nil. And given that the number of people who are actually willing to _test_ builds (as opposed to just bitch about how it doesn't do exactly what they want) is almost nil, I can't see this changing. more later, Anthony -- Anthony Baxter It's never too late to have a happy childhood. From guido@python.org Mon Apr 8 14:19:17 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 09:19:17 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 08:32:26 +0200." <0204080832260F.05787@arthur> References: <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> <0204080832260F.05787@arthur> Message-ID: <200204081319.g38DJHt11204@pcp742651pcs.reston01.va.comcast.net> > How does this amount of effort (keeping two CVS repositories in sync) > compare to maintaining just one tree but with "feature-test macros" as > one might try to do in C? I expect that most of the incompatibilities are the use of some tiny new shortcut in the language (think of +=, or list comprehensions, except that these were 2.0 features) where it really doesn't pay to do a version test -- instead, you end up coding in the old dialect. --Guido van Rossum (home page: http://www.python.org/~guido/) From mwh@python.net Mon Apr 8 14:17:26 2002 From: mwh@python.net (Michael Hudson) Date: 08 Apr 2002 14:17:26 +0100 Subject: [Python-Dev] checkins list seems to bo down... In-Reply-To: "M.-A. Lemburg"'s message of "Mon, 08 Apr 2002 14:06:45 +0200" References: <3CB187D5.AABC4B1B@lemburg.com> Message-ID: <2mwuvi49eh.fsf@starship.python.net> "M.-A. Lemburg" writes: > Subject says it all. Yes. > Any idea ? It may just be a delay (this happened last week). If messages are hitting the floor, I at least am Not Going To Be Happy. Cheers, M. -- Any form of evilness that can be detected without *too* much effort is worth it... I have no idea what kind of evil we're looking for here or how to detect is, so I can't answer yes or no. -- Guido Van Rossum, python-dev From guido@python.org Mon Apr 8 14:38:21 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 09:38:21 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 16:52:11 +1000." <200204080652.g386qBS06363@mbuna.arbhome.com.au> References: <200204080652.g386qBS06363@mbuna.arbhome.com.au> Message-ID: <200204081338.g38DcMo11747@pcp742651pcs.reston01.va.comcast.net> > Hm. I'd like to keep the "stable between 2.x.a and 2.x.b" thing going, > if only because the lib/python-x.y directory is shared between releases. For the stable track that should be guaranteed anyway. For the experimental track you shouldn't need this (users using the experimental track and wanting stability need to make up their mind). --Guido van Rossum (home page: http://www.python.org/~guido/) From David Abrahams" Message-ID: <012601c1df02$d70a4e30$6601a8c0@boostconsulting.com> OK. However, this will probably have to wait a month or so until it coincides with things in my development schedule that need it. That might be just as well since I take it Guido plans to change which fields are automatically inherited from base types... -Dave ----- Original Message ----- From: "Tim Peters" To: "David Abrahams" ; Sent: Monday, April 08, 2002 1:30 AM Subject: RE: [Python-Dev] GC, flags, and subtyping > [David Abrahams] > > ... > > P.S. One proviso: I find TeX to be cumbersome at best and hair-pullingly > > frustrating at worst. If I had to write TeX I probably would just submit > > code comprehensibility patches and stop there. I'm probably going to > > start with code patches anyway... > > Don't let LaTeX stand in your way: if you have useful docs to contribute, > plain text (and the plainer the better) works fine. Several people can > convert to proper Python LaTeX in their sleep, and I suspect that's what > Fred actually does in his sleep. I feel very comfortable making this offer, > since not one of the people who can do this even when awake. > > From guido@python.org Mon Apr 8 14:44:26 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 09:44:26 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 03:15:10 EDT." <15537.17278.333109.580542@anthem.wooz.org> References: <3CAEFC4E.2778AC55@python.org> <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <15537.11319.576212.846144@anthem.wooz.org> <15537.15490.631372.551394@12-248-41-177.client.attbi.com> <15537.17278.333109.580542@anthem.wooz.org> Message-ID: <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> > GvR> Or maybe 2.3 should become 2.2.3. <0.5 wink> > > BAW> I think the new bool type has already prevented that. > > SM> Why? If you postulate that 2.even.x become the experimental > SM> release branches, then 2.2.3 with a bool type makes perfect > SM> sense. > BAW> Given that interpretation, I'd agree. *If* we're going for an even/odd scheme, I wouldn't want to swap the Linux convention, so 2.2 would have to be stable, and 2.3 experimental. Given Alex's comments on 2.2, I don't think it's too far of a stretch to label 2.2 stable after the fact. It could make it easier to start experimenting with stuff in 2.3. I also like the idea of doing away with alpha/beta releases and use micro releases instead. I guess we'd have to issue a press release (we may already be doing one for 2.2.1, we may have to add some words to it). Maybe I should PEP it first? :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From mwh@python.net Mon Apr 8 14:46:02 2002 From: mwh@python.net (Michael Hudson) Date: 08 Apr 2002 14:46:02 +0100 Subject: [Python-Dev] 2.2.1 release mechanics In-Reply-To: Michael Hudson's message of "31 Mar 2002 21:39:24 +0100" References: <2m7knspj43.fsf@starship.python.net> Message-ID: <2mlmby9ucl.fsf@starship.python.net> Michael Hudson writes: > April 8 ~1200 GMT: freeze begins. > > Over the next 24 hours I'll do tests, write > Misc/NEWS, update version numbers, etc. This is now. Please make no checkins to the release22-maint branch unless you're an Appointed Wizard (me, Jack, Tim, Fred) or have been asked to by an Appointed Wizard. Especially given the -checkins situation. Cheers, M. -- Very clever implementation techniques are required to implement this insanity correctly and usefully, not to mention that code written with this feature used and abused east and west is exceptionally exciting to debug. -- Erik Naggum on Algol-style "call-by-name" From mwh@python.net Mon Apr 8 14:46:53 2002 From: mwh@python.net (Michael Hudson) Date: 08 Apr 2002 14:46:53 +0100 Subject: [Python-Dev] 2.2.1c2 RPMs In-Reply-To: Sean Reifschneider's message of "Tue, 2 Apr 2002 22:18:39 -0700" References: <20020402221838.G16962@tummy.com> Message-ID: <2mit729ub6.fsf@starship.python.net> Sean Reifschneider writes: > I've uploaded the RPMs for 2.2.1rc2. This is what the final 2.2.1 > beta will be based on, so if you have any final comments let me > know. Does this mean you have new spec files to check in? Cheers, M. -- If you don't have friends with whom to share links and conversation, you have social problems and you should confront them instead of joining a cultlike pseudo-community. -- "Quit Slashdot.org Today!" (http://www.cs.washington.edu/homes/klee/misc/slashdot.html#faq) From guido@python.org Mon Apr 8 14:53:59 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 09:53:59 -0400 Subject: [Python-Dev] GC, flags, and subtyping In-Reply-To: Your message of "Mon, 08 Apr 2002 08:37:32 CDT." <012601c1df02$d70a4e30$6601a8c0@boostconsulting.com> References: <012601c1df02$d70a4e30$6601a8c0@boostconsulting.com> Message-ID: <200204081353.g38Drx811869@pcp742651pcs.reston01.va.comcast.net> > OK. However, this will probably have to wait a month or so until it > coincides with things in my development schedule that need it. That > might be just as well since I take it Guido plans to change which fields > are automatically inherited from base types... I hope you're *happy* that I'm inheriting tp_is_gc and ob_type in 2.2.1 now. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 8 14:55:59 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 09:55:59 -0400 Subject: [Python-Dev] 2.2.1 release mechanics In-Reply-To: Your message of "08 Apr 2002 14:46:02 BST." <2mlmby9ucl.fsf@starship.python.net> References: <2m7knspj43.fsf@starship.python.net> <2mlmby9ucl.fsf@starship.python.net> Message-ID: <200204081355.g38DtxH11883@pcp742651pcs.reston01.va.comcast.net> > > April 8 ~1200 GMT: freeze begins. > > > > Over the next 24 hours I'll do tests, write > > Misc/NEWS, update version numbers, etc. > > This is now. Please make no checkins to the release22-maint branch > unless you're an Appointed Wizard (me, Jack, Tim, Fred) or have been > asked to by an Appointed Wizard. Note that I was just in time checking in the backward compatibility bool(), True and False builtins (returning ints, of course). If Anthony is listening, could I add those to 2.1.3 too? This will ease backporting 2.3 code to 2.1.3 and 2.2.1. > Especially given the -checkins situation. Sigh. I expect we're behind because SpamAssassin is working extra hours. :-( --Guido van Rossum (home page: http://www.python.org/~guido/) From aleax@aleax.it Mon Apr 8 14:53:00 2002 From: aleax@aleax.it (Alex Martelli) Date: Mon, 8 Apr 2002 15:53:00 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> References: <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> Message-ID: On Monday 08 April 2002 03:44 pm, Guido van Rossum wrote: > > GvR> Or maybe 2.3 should become 2.2.3. <0.5 wink> > > > > BAW> I think the new bool type has already prevented that. > > > > SM> Why? If you postulate that 2.even.x become the experimental > > SM> release branches, then 2.2.3 with a bool type makes perfect > > SM> sense. > > BAW> Given that interpretation, I'd agree. > > *If* we're going for an even/odd scheme, I wouldn't want to swap the > Linux convention, so 2.2 would have to be stable, and 2.3 > experimental. Given Alex's comments on 2.2, I don't think it's too > far of a stretch to label 2.2 stable after the fact. It could make it > easier to start experimenting with stuff in 2.3. Personally, I'd just _love_ this arrangement. 2.2.* stable, 2.3.* experimental -- and once you have a 2.3.x, for some value of x, that you judge IS worth becoming the next Stable branch, you can make it 2.4.0, or 2.4, or 3.0 -- whatever fits best. > I also like the idea of doing away with alpha/beta releases and use > micro releases instead. Yay for the Experimental branch -- not sure about the Stable branch, but we can think about at at the time 2.4 or whatever is ready; part of the arrangement should IMHO be to change the current commitment to twice-yearly-minor-releases into a "release early, release often" idea for Experimental, "release when needed to get important bug-fixes or backported stuff" for 2.Stable.microrelease -- but 2.(NextStable).0 should be solid when it happens (break compatibility with 2.PreviousStable.* only in specified ways, for example). Maybe that comes for free from 2.4.0 being == 2.3.something -- that depends on how popular the 2.3.* branch becomes, I guess. Hard to predict (offhand, it would seem to me that 2.3.* will be very popular, but I wouldn't be astonished if that proved to not be the case). > I guess we'd have to issue a press release > (we may already be doing one for 2.2.1, we may have to add some words > to it). Seems an opportunity (to send a nice-sounding message to a certain segment of the development community). > Maybe I should PEP it first? :-) Why not -- if you don't announce the PEP on c.l.p there shouldn't be too much troublesome bickering about it anyway:-). Alex From guido@python.org Mon Apr 8 15:01:17 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 10:01:17 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 22:25:23 +1000." <200204081225.g38CPNn10186@mbuna.arbhome.com.au> References: <200204081225.g38CPNn10186@mbuna.arbhome.com.au> Message-ID: <200204081401.g38E1Hu11915@pcp742651pcs.reston01.va.comcast.net> > > > Isn't this dangerous? Build config seems very delicate to me. > > Bloody hell yes. > > me too. I know I try to build releases on everything I can > get hold of (the cf.sf.net machines, the dec^Wdigital^Wcompaq^Wwhoever > testdrive boxes), but that's a whole lot of work. And that's not including > the, ahem, less unixish Unixes like AIX and HP/UX. Unless it's a platform > to which I have a lot of exposure (and in the last few years that's been > mostly linux and solaris), I'm not going to try and make this stuff work. > > > This is a real issue. There are build fixes for AIX 3 on sf for 221; > > they're not going in because I'm scared that they'll break something > > else. > > *nod* Yep. The chances of a build fix going in that's not completely > and utterly clearly the Right Thing to do is almost nil. And given that > the number of people who are actually willing to _test_ builds (as > opposed to just bitch about how it doesn't do exactly what they want) > is almost nil, I can't see this changing. All that is true, and yet, backporting portability might be important: as an old release ages, it's likely that new portability issues pop up, as new versions of Linux, AIX, HPUX appear. One thing that should be relatively easy is backporting the configure.in file. This is pretty much independent from the rest of Python; the only time when it wouldn't work would be if we stop defining a certain symbol because we don't need it any more. I expect that's very rare, because most symbols exist to deal with a specific portability issue, not with a specific Python feature. The testing is a problem, but if we can test 2.3 on platform X, it shouldn't be too hard to test 2.1.3 there, too. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Mon Apr 8 15:04:35 2002 From: skip@pobox.com (Skip Montanaro) Date: Mon, 8 Apr 2002 09:04:35 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204080652.g386qBS06363@mbuna.arbhome.com.au> References: <15537.15490.631372.551394@12-248-41-177.client.attbi.com> <200204080652.g386qBS06363@mbuna.arbhome.com.au> Message-ID: <15537.41843.997924.386742@12-248-41-177.client.attbi.com> >> Why? If you postulate that 2.even.x become the experimental release >> branches, then 2.2.3 with a bool type makes perfect sense. Once the >> bool type is in and you're satisfied that you've accounted for most >> of the necessary changes, you make a micro release. No big deal. >> Create tgz and zip files, maybe a Windows installer. On to the next >> one. Anthony> Hm. I'd like to keep the "stable between 2.x.a and 2.x.b" thing Anthony> going, if only because the lib/python-x.y directory is shared Anthony> between releases. Anthony, I don't know if I understand what you're getting at here. I think that 2.x.a -> 2.x.b should be relatively stable, certainly if x is odd. Less so if x is even, but most of the time not dramatically so. Let's postulate a Linux-like development process (at least w.r.t. CVS branches) and that the even-numbered minor numbers are the so-called experimental branches. Backing up briefly to fantasy land, we'd release 2.0.31, which we decree is "good enough". It becomes both 2.1.0 and 2.2.0. We announce a new "stable" release, 2.1.0 and start incorporating bug fixes. In reality, people will perhaps not start using it until its micro release number advances a bit, because the broader community will start hitting it harder and submitting more bug reports and fixes. Bugs that are fixed in the 2.1 branch obviously need to migrate across to the 2.2 branch as necessary. Enhancements applied to 2.2 would not migrate back to 2.1. I don't know if you can get away without a release champion. As you indicated, getting behind on backports can be a killer. In theory, most users and most Linux distributions will ignore the 2.2 branch and stick to 2.1 (those from whom we can pry 1.5.2 out of their cold, dead fingers). 2.1 releases will occur less frequently, almost entirely consist of just bug fixes, and should be "stable", which is what I think you're after. 2.2 releases will occur more frequently and may have a bit less stability between micro releases (bool type gets added to 2.2.3 and a bunch if is_* functions get corrected - in 2.2.4 more is_* functions get corrected, both of which may cause some minor breakage). People who can't tolerate a bit of shaky ground won't be messing around with this tree except to perhaps occasionally try it out. For this to work, a number of things have to happen. First, the effort required to actually cut a release have to drop dramatically. Even all the editing of files to replace 2.1.2 with 2.1.3 needs to be automated. (One way to accomplish this would be to have a standard patch file whose version numbers are twiddled, probably by a script, and which is then applied from the top of the source tree.) Getting from "let's cut a release tomorrow" to "2.1.4 is released" should not be much more labor than running "make dist" and sticking it on SF, at least on the Unix side of things. I don't know what's involved in making a Windows installer, but somebody besides Tim should be able to do that too. I presume that over time, the Mac will look more and more like Unix for distribution purposes as fewer and fewer people use Mac OS <= 9. Skip From guido@python.org Mon Apr 8 15:06:57 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 10:06:57 -0400 Subject: [Python-Dev] Pondering some changes to python.c... In-Reply-To: Your message of "Mon, 08 Apr 2002 10:33:21 +0200." <3CB155D1.422AD8C3@lemburg.com> References: <20020407042038.O16962@tummy.com> <3CB155D1.422AD8C3@lemburg.com> Message-ID: <200204081406.g38E6wD11990@pcp742651pcs.reston01.va.comcast.net> > -1 on making it the default -- there are programs relying on this. Agreed. > +1 on providing a sys module interface for this so that programs > can choose the name on their own. +1 --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Mon Apr 8 15:11:37 2002 From: skip@pobox.com (Skip Montanaro) Date: Mon, 8 Apr 2002 09:11:37 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> References: <3CAEFC4E.2778AC55@python.org> <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <15537.11319.576212.846144@anthem.wooz.org> <15537.15490.631372.551394@12-248-41-177.client.attbi.com> <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15537.42265.890383.842760@12-248-41-177.client.attbi.com> Guido> *If* we're going for an even/odd scheme, I wouldn't want to swap Guido> the Linux convention, so 2.2 would have to be stable, and 2.3 Guido> experimental. Yeah, I'd tend to agree. For one thing, all the Linux distributions are used to the Linux even-odd convention. Not confusing them would probably be a good thing. (Hey, how about we skip 2.3 altogether and get in complete version sync with Linux? Maybe we could even convince Linus to do our releases for us. :-) Guido> Maybe I should PEP it first? :-) You've got a smiley there, but some things will obviously have to change about the way releases are managed, so a PEP might not be a bad idea. Has Jack chimed in on this thread from the Mac quarter? Tim, can Windows installers be created by other people? Skip From mwh@python.net Mon Apr 8 15:17:14 2002 From: mwh@python.net (Michael Hudson) Date: 08 Apr 2002 15:17:14 +0100 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Misc NEWS,1.337.2.4.2.16,1.337.2.4.2.17 In-Reply-To: Guido van Rossum's message of "Fri, 05 Apr 2002 14:53:18 -0800" References: Message-ID: <2mbscul1g5.fsf@starship.python.net> Guido van Rossum writes: > Update of /cvsroot/python/python/dist/src/Misc > In directory usw-pr-cvs1:/tmp/cvs-serv5632 > > Modified Files: > Tag: release22-maint > NEWS > Log Message: > Some more news. (There's also a fix to _localemodule.c that I don't > dare describe, I don't think it merits a NEWS item. Cheers, M. From barry@zope.com Mon Apr 8 15:20:28 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Apr 2002 10:20:28 -0400 Subject: [Python-Dev] Historical backups? References: <20020407221021.GA2403@panix.com> <15536.54772.901785.884218@anthem.wooz.org> <20020407211648.P16962@tummy.com> <20020408083356.GX16278@xs4all.nl> Message-ID: <15537.42796.794391.242432@anthem.wooz.org> >>>>> "TW" == Thomas Wouters writes: TW> We do regular audits. The backups don't go back very far (a TW> week at most) but they do work. Thanks Thomas. -Barry From Anthony Baxter Mon Apr 8 15:20:26 2002 From: Anthony Baxter (Anthony Baxter) Date: Tue, 09 Apr 2002 00:20:26 +1000 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Message from Guido van Rossum of "Mon, 08 Apr 2002 10:01:17 -0400." <200204081401.g38E1Hu11915@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204081420.g38EKQG11509@mbuna.arbhome.com.au> >>> Guido van Rossum wrote > The testing is a problem, but if we can test 2.3 on platform X, it > shouldn't be too hard to test 2.1.3 there, too. In theory, yes. But in practice we get patches in the tracker that say "this makes the current python build on platform xyz, version N.M". Rarely will the person who contributed the patch also help with porting it to a different version of python, and equally rare is an explanation of _why_ the patch works. All that is left is to examine the patch, consult the chicken entrails, and hope. Anthony From guido@python.org Mon Apr 8 15:28:40 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 10:28:40 -0400 Subject: [Python-Dev] Array Enhancements In-Reply-To: Your message of "Mon, 08 Apr 2002 00:52:43 PDT." <20020408075243.79423.qmail@web12905.mail.yahoo.com> References: <20020408075243.79423.qmail@web12905.mail.yahoo.com> Message-ID: <200204081428.g38ESeL12108@pcp742651pcs.reston01.va.comcast.net> Scott Gilbert: > I hope the Numarray guys are doing a bang up job with their NDArray > type (I've looked at it briefly, but I don't really understand it > yet...). I suspect that most of the ufuncs and other stuff those guys > are doing are too special purpose to be part of the standard Python > baseline, but I would very much like to see a single usable array type > become the standard. I'd be willing to do PEP grunt work for that. The Numarray fols already have a PEP. I'd only be willing to consider another array module if it was a clean subset or baset class of the full Numarray functionality; I think the Numarray folks are thinking about that already. (This would mean that the Numarray base type would be part of core Python but that the Numarray folks would maintain their own ufuncs library on top of it.) > It's not that much bloat. It would be a setitem and getitem pair for > each new type. If Tim says it's code bloat, it's code bloat. Don't argue. :-) > Did you guys really make it possible to unpickle a Unicode string in > versions of Python that were pre Unicode? No, but other than that *addition* the pickle format has been pretty stable. And I just checked in code that ensures that bools pickled by 2.3 will be unpicklable as ints in 2.2 and before (without even changing the 2.2 unpickler!) > I would think new features should only work in new versions... I understand that that's what you think, and that's why we have to keep explaining to you that for pickling you need to be more conservative. Pickling is often used as a client-server mechanism, and you don't always (want to) have control over the Python version used on the other end. > That must have snuck in there sometime after 2.2 I guess. Yes, the array module has been overhauled significantly. Please always use current CVS. > > But it's a useful thing to be able to do, I agree, and it > > shouldn't be too hard to add a flag that says "I don't own this > > memory" -- which would mean that the buffer can't be resized at > > all. > > I pictured this working like CObjects do where you pass in a destructor > for when the reference count goes to zero. Possibly also passing in a > realloc function. If the realloc function is null, then an exception > is raised when someone tries to resize the array. That'll work too. Tim already pointed out the other problem -- the C code that *does* own the memory must know if you still have a reference to it. > This means there would need to be a C visible API for building array > objects around special types of memory though. That's OK. Just design one. > Guido also wrote: > > Since arrays are all about compromises that trade flexibility for > > speed and memory footprint, you can't have a one size fits all. :-) > > Bahh. I don't think getting a good general purpose Python object that > represents arbitrary C arrays is all that impossible. C arrays just > don't do that much. I'll bet you 10 dollars that your patch will be larger than the code it patches, if you really implement everything on your wish list. > Besides I didn't say "one size fits all", I said "one size fits all my > needs". That "my" is important (at least to me :-) But you did offer to make it a standard Python module, so you'll have to deal with other people's wishes too. It seems to me that you're pretty green at this... :-( > Guido also wrote: > > > Well if someone authoritative tells me that all of the above is a > > > great idea, I'll start working on a patch and scratch my plans to > > > create a "not in house" xarray module. [I didn't write that, you did; I wrote this:] > > It all depends on the quality of the patch. By the time you're done > > you may have completely rewritten the array module, and then the > > question is, wouldn't your own xarray module have been quicker to > > implement, because it doesn't need to preserve backwards > > compatibility? > > Yup, I think I would be done with my xarray module by now if I had > written it instead of taking this route. It would also have the > disadvantage that it doesn't play nice with anyone else. But you said you only cared about your own needs? :-) > I now think the best bet is to replace the array module with something > flexible enough to: > > 1) do what it currently does And that's where backwards compatibility is going to kill most of your innovation. Consider yourself lucky that the array module currently doesn't define any C APIs, so at least there aren't any C APIs that need to keep working... This is one of the Numarray nightmares. > 2) do what the Numarray guys need > 3) do what I need Sounds like working from some common base of Numarray makes more sense. > Guido also wrote: > > An alternative might be a separate bit-array implementation: it seems > > that the bit-array won't share much code with the regular array (of > > any flavor), so why not make it a separate type? > > Yup. It would be nice if a bitarray was actually the same type, Why? Have you really thought this through? It can still have the same constructor function, but why would being the same type make any difference? > but having code like: > > if (o->is_bitarray) { > /* do something */ > } else { > /* do every other byte addressable type */ > } > > is a little ugly. In fact, it's why OO programming was invented. No kidding! > Well, part of my preference for modifying arraymodule.c instead of > Numarray is that I very quickly understood what's going on in > arraymodule.c, and a patch is pretty obvious. Looking at Numarray, I > just don't get it yet. Please take this as a shortcoming in my > abilities. Numarray does appear to be the heir-apparent though, so > I'll give it a better look. Yes, please do. (And if you conclude that Numarray stinks, please write down a careful review and discuss it both with the Numarray authors and with us -- we don't want our heir to be dead in the throne. :-) > I also assumed that the Numarray folks would play nice with the > standard array module. I don't think that's one of their requirements, and I don't see why it should be. > David also wrote: > > > > I'd like to see fewer multi-dimensional array objects, not more... > > I agree completely. In fact, I'd like to see one official one > distributed with the baseline. Eventually, this will be Numarray, most likely. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 8 15:33:57 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 10:33:57 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 09:11:37 CDT." <15537.42265.890383.842760@12-248-41-177.client.attbi.com> References: <3CAEFC4E.2778AC55@python.org> <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <15537.11319.576212.846144@anthem.wooz.org> <15537.15490.631372.551394@12-248-41-177.client.attbi.com> <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> <15537.42265.890383.842760@12-248-41-177.client.attbi.com> Message-ID: <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> > Guido> *If* we're going for an even/odd scheme, I wouldn't want to swap > Guido> the Linux convention, so 2.2 would have to be stable, and 2.3 > Guido> experimental. > [Skip] > Yeah, I'd tend to agree. For one thing, all the Linux distributions > are used to the Linux even-odd convention. Not confusing them would > probably be a good thing. (Hey, how about we skip 2.3 altogether > and get in complete version sync with Linux? Maybe we could even > convince Linus to do our releases for us. :-) I think Linus has once said that Python alone was not a reason to reject a certain piece of software; that's a far cry from helping us out. :-) > Guido> Maybe I should PEP it first? :-) > > You've got a smiley there, Because any PEP I write appears to mostly raise controversy once brought up on c.l.py. > but some things will obviously have to change about the way releases > are managed, so a PEP might not be a bad idea. An informational PEP explaining the procedure, yes. A PEP asking the community what they want, heck no. The ones that are still believing that python-dev is a closed list should *definitely* continue to believe so. The fake toll booth is doing its work admirably. :-) > Has Jack chimed in on this thread from the Mac quarter? Tim, can > Windows installers be created by other people? --Guido van Rossum (home page: http://www.python.org/~guido/) From Anthony Baxter Mon Apr 8 15:34:11 2002 From: Anthony Baxter (Anthony Baxter) Date: Tue, 09 Apr 2002 00:34:11 +1000 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Message from Skip Montanaro of "Mon, 08 Apr 2002 09:04:35 EST." <15537.41843.997924.386742@12-248-41-177.client.attbi.com> Message-ID: <200204081434.g38EYBP11707@mbuna.arbhome.com.au> >>> Skip Montanaro wrote > Let's postulate a Linux-like development process (at least w.r.t. CVS > branches) and that the even-numbered minor numbers are the so-called > experimental branches. On thinking about it some more, I don't think I like the "even == experimental, odd == stable" approach. It's way too mystical and non- obvious. All too much magic, and particularly if we say that "2.odd.n -> 2.odd.n+1 is guaranteed to be stable, but 2.even.n -> 2.even.n+1, all bets are off". I'm not sure I can think of a better approach, unfortunately, unless we do something like "the release 2.n will be supported for at least {X minor revisions of 2.n+1, X months after 2.n+1.0, ....}", and have something like that. So 2.1.x is "stable" until after 2.2.1 is out and happy. In practice, this is what happens anyway - I know I'm not running anything I really care about on 2.2 yet. I'm also not so keen on the version-itis that I think having multiple active branches will lead to. I look at Perl and I go "ick". It seems every time I want to install a perl package, I have to go through a painful exercise in upgrading half the damn cpan-installed packages on my system - and every time I've tried to use the 'auto-follow dependencies', I've come back to find it's downloaded the latest and greatest point release of the perl core and is building it, because some package requires the most bleeding edge features. I don't _want_ to have to play that game in python. > For this to work, a number of things have to happen. First, the effort > required to actually cut a release have to drop dramatically. Even all the > editing of files to replace 2.1.2 with 2.1.3 needs to be automated. The editing of files is completely and utterly the least of the problems. It's mostly a matter of pushing the right buttons on the right web interfaces, that sort of thing. Now while this could be automated, there's not a lot of point. Since 2.1.2, the sourceforge interfaces for doing a file release have changed, slightly. Not so much that a human can't figure it out, but it'd stuff any automated systems completely. > Getting from "let's cut a release tomorrow" to > "2.1.4 is released" should not be much more labor than running "make dist" > and sticking it on SF, at least on the Unix side of things. I wish! Have a read of PEP 102. > I don't know > what's involved in making a Windows installer, but somebody besides Tim > should be able to do that too. I assume nothing. -- Anthony Baxter It's never too late to have a happy childhood. From fdrake@acm.org Mon Apr 8 15:35:59 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 8 Apr 2002 10:35:59 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15537.12457.59929.194685@anthem.wooz.org> References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> Message-ID: <15537.43727.369744.356521@grendel.zope.com> Barry A. Warsaw writes: > And of course, I spend a non-trivial amount of effort to maintain a > separate email package that is compatible with Python 2.1 and Python > 2.2. But I consciously gave up Python 2.0 (or earlier) support > because it became obvious to me that I wanted /some/ of the newer > features. PyXML is specifically maintaining compatibility, which means I have to work around the lack of weakrefs when removing reference cycles. The differences between the Python standard library and PyXML, for modules present in both, is mostly trivial still, but the differences are growing. ;-( (It's enough to make me think that Andrew may have been right about not integrating the XML support the way we did.) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From fdrake@acm.org Mon Apr 8 15:38:35 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 8 Apr 2002 10:38:35 -0400 Subject: [Python-Dev] GC, flags, and subtyping In-Reply-To: References: <09ca01c1dc5f$18a16a00$1101a8c0@boostconsulting.com> Message-ID: <15537.43883.895318.619736@grendel.zope.com> Tim Peters writes: > convert to proper Python LaTeX in their sleep, and I suspect that's what > Fred actually does in his sleep. Hey, if I didn't do that, it would just be wasted time, and we can't have that, can we? ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From guido@python.org Mon Apr 8 15:44:06 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 10:44:06 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 09:04:35 CDT." <15537.41843.997924.386742@12-248-41-177.client.attbi.com> References: <15537.15490.631372.551394@12-248-41-177.client.attbi.com> <200204080652.g386qBS06363@mbuna.arbhome.com.au> <15537.41843.997924.386742@12-248-41-177.client.attbi.com> Message-ID: <200204081444.g38Ei6V12247@pcp742651pcs.reston01.va.comcast.net> > I don't know if I understand what you're getting at here. I think > that 2.x.a -> 2.x.b should be relatively stable, certainly if x is > odd. Less so if x is even, but most of the time not dramatically > so. Ehm, haven't you got that backwards? I did "uname -a" and my kernel version is 2.4.9, which would suggest that even minor numbers are stable for Linux. And you just agreed that we should do the same for Python. > For this to work, a number of things have to happen. First, the > effort required to actually cut a release have to drop dramatically. Yes, I think that our current process is a bit too heavy. I guess we should stop having a new webpage for each release, and the Doc, Mac and Win releases could be completely from the source releases (proceeding at their own, usually slower, pace). > Even all the editing of files to replace 2.1.2 with 2.1.3 needs to > be automated. At least the LICENSE file should stop referring to a specific micro version. Lawyers be damned. > (One way to accomplish this would be to have a standard patch file > whose version numbers are twiddled, probably by a script, and which > is then applied from the top of the source tree.) You'd still have to watch it though. > Getting from "let's cut a release tomorrow" to "2.1.4 is released" > should not be much more labor than running "make dist" and sticking > it on SF, at least on the Unix side of things. Let's also forget about releasing via SF (at least for experimental releases). It's slow and cumbersome, and most people download from python.org anyway. > I don't know what's involved in making a Windows installer, but > somebody besides Tim should be able to do that too. I can do it in 5 minutes, but I don't bother to test it much. Again, for an experimental release that shouldn't be a big deal. > I presume that over time, the Mac will look more and more like Unix > for distribution purposes as fewer and fewer people use Mac OS <= 9. Certainly they might not be interested in experimental releases much (you have to conclude that people still running Mac OS <= 9 are not early adopters ;-). --Guido van Rossum (home page: http://www.python.org/~guido/) From Anthony Baxter Mon Apr 8 15:42:51 2002 From: Anthony Baxter (Anthony Baxter) Date: Tue, 09 Apr 2002 00:42:51 +1000 Subject: [Python-Dev] 2.2.1 release mechanics In-Reply-To: Message from Guido van Rossum of "Mon, 08 Apr 2002 09:55:59 -0400." <200204081355.g38DtxH11883@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204081442.g38Egpe11870@mbuna.arbhome.com.au> >>> Guido van Rossum wrote > Note that I was just in time checking in the backward compatibility > bool(), True and False builtins (returning ints, of course). If > Anthony is listening, could I add those to 2.1.3 too? This will ease > backporting 2.3 code to 2.1.3 and 2.2.1. I can't decide on whether this should go in 2.1.3 or not. On the plus side, it's unlikely to break things. On the minus side, it is breaking the "contract"[*] about what goes into a minor release. [*] where "contract" is between the python developers and the people who install and use it. up until now, we've said 'minor releases are bugfixes only' - changing this at the last minute before a release feels a bit wrong, somehow. Given the nature of the bug fixes in 2.1.3, I _really_ don't want to give people _any_ fears that they might have backwards compatability concerns to deal with. I can't see any way that adding bool/True/False to 2.1.3 will break things, but it's a perceptions thing, I guess. I just want to see 2.1, 2.1.1 and 2.1.2 go away :) Hm. At this point (well, at the point I got Guido's email), the 2.1.3 release is pretty much done but for . make install on creosote. . rename the file in ftp.python.org (it's called rc0 right now). . pushing the 'hidden->active' button on sf. . sending the email announcement. If I'm going to add this, I'll need to recut the release. In any case, I'm not going to do this right now (it's late) - I'll look at it first thing tomorrow and try to decide then. I really _would_ like feedback, though. Anthony From guido@python.org Mon Apr 8 15:46:11 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 10:46:11 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 10:35:59 EDT." <15537.43727.369744.356521@grendel.zope.com> References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> <15537.43727.369744.356521@grendel.zope.com> Message-ID: <200204081446.g38EkC312270@pcp742651pcs.reston01.va.comcast.net> > PyXML is specifically maintaining compatibility, which means I have to > work around the lack of weakrefs when removing reference cycles. Weren't weakrefs an external module in 2.1? Can't you distribute that separately with PyXML (under a different name)? > (It's enough to make me think that Andrew may have been right about > not integrating the XML support the way we did.) Yeah, but there were some very loud supporters who mostly wanted it for political reasons at the time. I think I may have fallen for their arguments a little too fast. ;-) --Guido van Rossum (home page: http://www.python.org/~guido/) From petrilli@amber.org Mon Apr 8 16:05:53 2002 From: petrilli@amber.org (Christopher Petrilli) Date: Mon, 08 Apr 2002 11:05:53 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15537.41843.997924.386742@12-248-41-177.client.attbi.com> Message-ID: Maybe I'm getting old, having grown up under BSD4.2, but I've continually found the Linux approach to be mind-boggling confusing, and approaching some hacker's fantasy environment. While this is great for a fast moving kernel that tries to be everything to everyone, I find it difficult to keep track of the interdependencies of patches and releases, and it's just simply not obvious to me that because something an odd number that it should necessarily be of lower quality than something of an even number. I don't believe that the alpha-beta-gamma-candidate-release process has developed out of thin air at hundreds of organizations, and it certainly has a familiarity that to me brings with it a more serious organization. What troubles me about these discussions is the following, based on what I've run into in working with customers who have lots of Linux boxes. 1) Those people who want to be on the bleeding edge do not, by nature, care what release numbers mean. They'll work out of CVS repositories, or the similar "snapshot" systems, and can not be convinced otherwise. 2) Those people who care about stability do not, by nature, update frequently, and in fact are often put off by frequent releases. To them, the more frequent the release, the more indicative it is of bad coding, or instability. We can not, while maintaining our collective sanity, please both sets of people. Traditionally, Python has appealed to the 2nd set because of its very deliberative nature in the process of adding new features. The PEP process has only further rationalized what was unofficial before, and I applaud its structured approach to progress. This appeals to a large group of people who want a stable, deliberate progression of releases. For example, the FreeBSD/OpenBSD projects both use a very similar approach in that they have major/minor releases, with the following conventions in their numbering scheme: X.Y.Z for the following... X = Major release, with allowed substantial backward incompatibility Y = Feature release, minimal backward incompatibility, with everything done to avoid these issues. Z = Bug fixes only. These may break people who have "worked around" the issue, but shouldn't affect 99.999% of the population. The first tenant to releases is that you can not please everyone all of the time, most especially people who pay nothing for what they receive. We can make every reasonable effort to avoid backward incompatibility, but forward progress necessitates the incompatibility at some point. The key to avoiding the pain is to introduce it in stages, that one might propose follows the timeline, with the current release X.Y: 1) Propose change, formalize specification 2) Approve change after review and inclusion of feedback 3) Publish change in a prominent location 4) Next release X.(Y+1) provides support, but in the __future__ mode, and only warns for existing incompatible usage. 5) Follow on release X.(Y+2..n) provides for the new state as the default, but allows for the inclusion of old behavior via __past__. 6) Next release (X+1).Y drops all support for the previous syntax/cap. and forces the user to change. This will allow people who want to migrate to future release without upgrading huge code bases to continue to do so. One might even envision a site configuration option that would allow users to set the default behavior, but then what of those who assume differently? I will point out that it is trivial to maintain multiple X.Y releases on the same machine, and for example, I have 1.5.2, 2.1 and 2.2 on my main development machine currently, and simply always make sure to specify the Python release in the #! header at the top of the file. This way in the future, I can still use it. A panacea of universal happiness is a wonderful thing, but I think we need to step back and answer the question of who we are trying to please with these proposed changes, whether that group is in fact using Python at any large level, or would use Python at any large level, and whether or not we would alienate people who have been with the community for a large period of time. As someone commented (and I can't remember who now), this needs to be taken in the context of increasing the spread of Python, rather than pleasing a potentially vocal minority. If such a change is advantageous to the spread of Python and its adoption, then I'm 100% for it. But if it simply cow-tows to "peer pressure," then it seems to me that it forsakes progress on the alter of change. Please remember that change and progress are not synonyms. Chris -- | Christopher Petrilli From fdrake@acm.org Mon Apr 8 16:09:05 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 8 Apr 2002 11:09:05 -0400 Subject: [Python-Dev] GC, flags, and subtyping In-Reply-To: <012601c1df02$d70a4e30$6601a8c0@boostconsulting.com> References: <012601c1df02$d70a4e30$6601a8c0@boostconsulting.com> Message-ID: <15537.45713.562532.891683@grendel.zope.com> David Abrahams writes: > OK. However, this will probably have to wait a month or so until it > coincides with things in my development schedule that need it. That > might be just as well since I take it Guido plans to change which fields > are automatically inherited from base types... In the meanwhile, you should see a lot of progress. A great way to help out without having to spend a lot of time would be to read what's been added periodically and let us know (via the SF bug tracker) whether there were questions you expected to be answered with the addition that went unanswered. Or reporting clarity failures & the like as well, or course. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From skip@pobox.com Mon Apr 8 16:14:41 2002 From: skip@pobox.com (Skip Montanaro) Date: Mon, 8 Apr 2002 10:14:41 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081434.g38EYBP11707@mbuna.arbhome.com.au> References: <15537.41843.997924.386742@12-248-41-177.client.attbi.com> <200204081434.g38EYBP11707@mbuna.arbhome.com.au> Message-ID: <15537.46049.729997.710611@12-248-41-177.client.attbi.com> >> Let's postulate a Linux-like development process (at least w.r.t. CVS >> branches) and that the even-numbered minor numbers are the so-called >> experimental branches. Anthony> On thinking about it some more, I don't think I like the "even Anthony> == experimental, odd == stable" approach. I only postulated that because people seem to have been calling "2.1.x" the Garth branch. I don't really care one way or the other, and having the same even-odd meaning as the Linux kernel means you won't fight the tide of people who already know that way of doing things. Anthony> I'm also not so keen on the version-itis that I think having Anthony> multiple active branches will lead to. I look at Perl and I go Anthony> "ick". So do I, but generally for different reasons. :-) I don't think the transition from 2.1.3 to 2.1.4 would have the same weight and corresponding need for attention that it does today. As an example, consider that serious bug that was just discovered and fixed. Once checked in and tested, Guido should have been able to say, "okay, time for a point release", after which a "few buttons get pushed" (hand wave, hand wave) and a new release appears on SF. >> For this to work, a number of things have to happen. First, the >> effort required to actually cut a release have to drop dramatically. >> Even all the editing of files to replace 2.1.2 with 2.1.3 needs to be >> automated. Anthony> The editing of files is completely and utterly the least of the Anthony> problems. It's mostly a matter of pushing the right buttons on Anthony> the right web interfaces, that sort of thing. Now while this Anthony> could be automated, there's not a lot of point. Since 2.1.2, Anthony> the sourceforge interfaces for doing a file release have Anthony> changed, slightly. Not so much that a human can't figure it Anthony> out, but it'd stuff any automated systems completely. That wasn't what I was referring to. I was referring to the spate of checkins where people have to advance version numbers prior to a release. I would hope that could be automated. >> Getting from "let's cut a release tomorrow" to "2.1.4 is released" >> should not be much more labor than running "make dist" and sticking >> it on SF, at least on the Unix side of things. Anthony> I wish! Have a read of PEP 102. Obviously, if what I have in mind is going to work, PEP 102 is going to have to be streamlined quite a bit. Maybe that's not possible. If that turns out to be a significant barrier, the notion of "lighter weight" releases probably isn't going to work. >> I don't know what's involved in making a Windows installer, but >> somebody besides Tim should be able to do that too. Anthony> I assume nothing. The fact that Tim owns that process and nobody else seems willing to pick it up suggests otherwise. Skip From Anthony Baxter Mon Apr 8 16:28:44 2002 From: Anthony Baxter (Anthony Baxter) Date: Tue, 09 Apr 2002 01:28:44 +1000 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Message from Skip Montanaro of "Mon, 08 Apr 2002 10:14:41 EST." <15537.46049.729997.710611@12-248-41-177.client.attbi.com> Message-ID: <200204081528.g38FSi112614@mbuna.arbhome.com.au> > I only postulated that because people seem to have been calling "2.1.x" > the Garth branch. I don't really care one way or the other, and having the > same even-odd meaning as the Linux kernel means you won't fight the tide of > people who already know that way of doing things. That tide's only the linux users, and the linux users who care what kernel they're running. I'm an ex-sysadmin, pretty solid unix geek, and I nowadays run whatever my distro comes with - as long as it supports what I want to do, I don't care. I don't think that saying "it's what linux does" is enough to carry the day by itself. I suspect that if you want to go with the flow, we should follow the approach that Microsoft uses to mark their unstable releases. (And I'm _not_ going to use any one of the many, many cheap shots about unstable MS releases that I have in my head here - feel free to substitute your own. :) > Once checked in and tested, Guido > should have been able to say, "okay, time for a point release", after which > a "few buttons get pushed" (hand wave, hand wave) and a new release appears > on SF. Hm. Maybe. It would be nice, but the other thing to consider it that particularly for a "Garth" branch (does that mean the experimental branch gets calle Wayne, I wonder?) you want to try to build it on a couple of boxes first - just in case. > That wasn't what I was referring to. I was referring to the spate of > checkins where people have to advance version numbers prior to a release. I > would hope that could be automated. That's pretty much down to Include/patchlevel.h - two spots README - one line Misc/NEWS - if it's been kept up to date, no changes. > Obviously, if what I have in mind is going to work, PEP 102 is going to have > to be streamlined quite a bit. No complaints from me :) > >> I don't know what's involved in making a Windows installer, but > >> somebody besides Tim should be able to do that too. > Anthony> I assume nothing. > The fact that Tim owns that process and nobody else seems willing to pick it > up suggests otherwise. I should be clearer - I seem to recall during 2.1.2 that Tim muttered something about 'half a day' to do the release. He also muttered something about 'damn bass players' too, though, so I'm not sure how realistic he was being there... Anthony From David Abrahams" <012601c1df02$d70a4e30$6601a8c0@boostconsulting.com> <15537.45713.562532.891683@grendel.zope.com> Message-ID: <021001c1df12$09427440$6601a8c0@boostconsulting.com> ----- Original Message ----- From: "Fred L. Drake, Jr." > In the meanwhile, you should see a lot of progress. A great way to > help out without having to spend a lot of time would be to read what's > been added periodically and let us know (via the SF bug tracker) > whether there were questions you expected to be answered with the > addition that went unanswered. Or reporting clarity failures & the > like as well, or course. Just tell me where to look for progress... -Dave From Anthony Baxter Mon Apr 8 16:32:11 2002 From: Anthony Baxter (Anthony Baxter) Date: Tue, 09 Apr 2002 01:32:11 +1000 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Message from Christopher Petrilli of "Mon, 08 Apr 2002 11:05:53 -0400." Message-ID: <200204081532.g38FWBc12664@mbuna.arbhome.com.au> >>> Christopher Petrilli wrote > [some good stuff] one last thing before I head off to sleep - it seems to me that one of the groups we really want to make sure we support is the people doing the development of python. If making lots of micro- releases is going to lead to far more triage work and chasing up bug reports, then that's a bad bad thing. Anthony From fdrake@acm.org Mon Apr 8 16:37:42 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 8 Apr 2002 11:37:42 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081446.g38EkC312270@pcp742651pcs.reston01.va.comcast.net> References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> <15537.43727.369744.356521@grendel.zope.com> <200204081446.g38EkC312270@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15537.47430.580564.238632@grendel.zope.com> Guido van Rossum writes: > Weren't weakrefs an external module in 2.1? Can't you distribute that > separately with PyXML (under a different name)? They were an external module in that the types as well as the Python-level access was in Modules/_weakref.c, but invalidating references required cooperation from the core, since action is taken when the refcount goes to 0. It cannot work for 2.0 under any name. > Yeah, but there were some very loud supporters who mostly wanted it > for political reasons at the time. I think I may have fallen for > their arguments a little too fast. ;-) Maybe. I *still* think providing XML support out of the box is a good thing, the questions for me are related to how that should be done rather than whether it should be. But that's a separate discussion. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From barry@zope.com Mon Apr 8 16:38:19 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Apr 2002 11:38:19 -0400 Subject: [Python-Dev] Re: Stability and change References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> <15537.43727.369744.356521@grendel.zope.com> Message-ID: <15537.47467.132247.414879@anthem.wooz.org> >>>>> "Fred" == Fred L Drake, Jr writes: Fred> (It's enough to make me think that Andrew may have been Fred> right about not integrating the XML support the way we did.) I wonder if decoupling more of the standard library isn't a good idea, although it flies in the face of "batteries included". One thing I like about XEmacs's model is that I kind of get both for little effort. I want all the latest packages, but I don't want to re-install XEmacs to get them. So I use the package manager, which isn't exactly one-click-updates, but it's pretty simple to use. We'd still have to deal with incompatibilities in core modules, and in the language spec, but those probably require different tools. -Barry From skip@pobox.com Mon Apr 8 16:55:24 2002 From: skip@pobox.com (Skip Montanaro) Date: Mon, 8 Apr 2002 10:55:24 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081528.g38FSi112614@mbuna.arbhome.com.au> References: <15537.46049.729997.710611@12-248-41-177.client.attbi.com> <200204081528.g38FSi112614@mbuna.arbhome.com.au> Message-ID: <15537.48492.167677.298169@12-248-41-177.client.attbi.com> Anthony> That tide's only the linux users, and the linux users who care Anthony> what kernel they're running. I'm an ex-sysadmin, pretty solid Anthony> unix geek, and I nowadays run whatever my distro comes with - Yup, me too. I was mostly thinking about the Linux distro folks themselves. It makes it a bit easier for them if they think, "Ah, yes, 2.3 is an experimental release. We better target Pythonn 2.2 or 2.4 for our next release." >> That wasn't what I was referring to. I was referring to the spate of >> checkins where people have to advance version numbers prior to a >> release. I would hope that could be automated. Anthony> That's pretty much down to Anthony> Include/patchlevel.h - two spots Anthony> README - one line Anthony> Misc/NEWS - if it's been kept up to date, no changes. I don't know. I see a lot more version-related checkins than that float by just before a release. What about the docs for example? S From guido@python.org Mon Apr 8 17:07:44 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 12:07:44 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 11:05:53 EDT." References: Message-ID: <200204081607.g38G7iD12531@pcp742651pcs.reston01.va.comcast.net> > 1) Those people who want to be on the bleeding edge do not, by > nature, care what release numbers mean. They'll work out of > CVS repositories, or the similar "snapshot" systems, and can > not be convinced otherwise. In Python's case, that's a vanishingly small fraction of the total user base. Even the alpha/beta releases are downloaded by a disappointingly small number of people. Many Python users apparently consider themselves early adopters when they install 2.2 final when it comes out. > 2) Those people who care about stability do not, by nature, > update frequently, and in fact are often put off by frequent > releases. To them, the more frequent the release, the more > indicative it is of bad coding, or instability. Unfortunately, what's frequent for one person appears at glacial speed for another. Since 2.0 came out, we've new Python releases (counting only the minor version numbers) every 6-8 months. Apparently the conservative forces in the community find this too often. > We can not, while maintaining our collective sanity, please both > sets of people. Traditionally, Python has appealed to the 2nd set > because of its very deliberative nature in the process of adding new > features. The PEP process has only further rationalized what was > unofficial before, and I applaud its structured approach to > progress. This appeals to a large group of people who want a > stable, deliberate progression of releases. And yet, there is clamoring in the newsgroup that Python changes too fast, that we add features to minor releases that should deserve a major release, and that a minor release every 6 months is too often. So apparently we're not doing enough for these people. I guess that it means that Python is becoming more popular -- as your audience grows, it necessarily becomes more conservative (there just aren't enough early adopters in the world). > For example, the FreeBSD/OpenBSD projects both use a very similar > approach in that they have major/minor releases, with the following > conventions in their numbering scheme: > > X.Y.Z for the following... > > X = Major release, with allowed substantial backward > incompatibility > Y = Feature release, minimal backward incompatibility, with > everything done to avoid these issues. > Z = Bug fixes only. These may break people who have "worked around" > the issue, but shouldn't affect 99.999% of the population. That's what I've always thought the Python version numbering meant too. The argument appears to be mostly in the definition of "minimal backward incompatibility". *I* though that PEP 285 did everything humanly possible to make the bool type backward compatible. But according to the c.l.py crowd it's not enough. > The first tenant to releases is that you can not please everyone all > of the time, most especially people who pay nothing for what they > receive. We can make every reasonable effort to avoid backward > incompatibility, but forward progress necessitates the > incompatibility at some point. Try telling Logajan or Rubin that. :-( > The key to avoiding the pain is to introduce it in stages, that one > might propose follows the timeline, with the current release X.Y: > > 1) Propose change, formalize specification > 2) Approve change after review and inclusion of feedback > 3) Publish change in a prominent location We do all that. > 4) Next release X.(Y+1) provides support, but in the __future__ > mode, and only warns for existing incompatible usage. It depends on the nature of the feature if that's possible. A new type doesn't stay confined to modules that use the future statement; whenever an instance of it is returned, there's a possibility that the receiver of that instance is not prepared to handle it. I can't make str(True) return something different based on whether the caller has executed a __future__ statement -- it just wouldn't do the right thing in too many cases. > 5) Follow on release X.(Y+2..n) provides for the new state as > the default, but allows for the inclusion of old behavior via > __past__. > 6) Next release (X+1).Y drops all support for the previous > syntax/cap. and forces the user to change. Right on. > This will allow people who want to migrate to future release without > upgrading huge code bases to continue to do so. I sure hope so, but in practice, even the smallest incompatibility between 2.Y and 2.(Y+1) seems to trip people up. > One might even envision a site configuration option that would allow > users to set the default behavior, but then what of those who assume > differently? Have multiple versions installed, with binaries named "python2.1, python2.2, python2.3" and "python" linked to the most popular one. Bleeding edge users can place a symlink to python2.3 in their personal bin directory if they don't want to type "python2.3" all the time. > I will point out that it is trivial to maintain multiple X.Y > releases on the same machine, and for example, I have 1.5.2, 2.1 and > 2.2 on my main development machine currently, and simply always make > sure to specify the Python release in the #! header at the top of > the file. This way in the future, I can still use it. Apparently few people know how to use this properly. Also, there are those who want the impossible: they want no old code to break, yet they want to be able to use bleeding edge packages. > A panacea of universal happiness is a wonderful thing, but I think > we need to step back and answer the question of who we are trying to > please with these proposed changes, whether that group is in fact > using Python at any large level, or would use Python at any large > level, and whether or not we would alienate people who have been > with the community for a large period of time. This certainly seems closer to resolution that peace in the Middle East. :-) > As someone commented (and I can't remember who now), this needs to > be taken in the context of increasing the spread of Python, rather > than pleasing a potentially vocal minority. If such a change is > advantageous to the spread of Python and its adoption, then I'm 100% > for it. Unfortunately, the vocal minority is claiming that Python's fast-paced releases are what keeps the spread from going up. Are they right? Who knows? > But if it simply cow-tows to "peer pressure," then it seems > to me that it forsakes progress on the alter of change. Please > remember that change and progress are not synonyms. All in all, I don't think your post has told me anything new. --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Mon Apr 8 17:07:41 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Apr 2002 12:07:41 -0400 Subject: [Python-Dev] 2.2.1 release mechanics References: <200204081355.g38DtxH11883@pcp742651pcs.reston01.va.comcast.net> <200204081442.g38Egpe11870@mbuna.arbhome.com.au> Message-ID: <15537.49229.491064.702359@anthem.wooz.org> >>>>> "AB" == Anthony Baxter writes: AB> Given the nature of the bug fixes in 2.1.3, I _really_ don't AB> want to give people _any_ fears that they might have backwards AB> compatability concerns to deal with. I can't see any way that AB> adding bool/True/False to 2.1.3 will break things, but it's a AB> perceptions thing, I guess. --------^^^^^^^^^^^^^^^^^ I agree. -1 on adding them. -Barry From guido@python.org Mon Apr 8 17:16:05 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 12:16:05 -0400 Subject: [Python-Dev] 2.2.1 release mechanics In-Reply-To: Your message of "Tue, 09 Apr 2002 00:42:51 +1000." <200204081442.g38Egpe11870@mbuna.arbhome.com.au> References: <200204081442.g38Egpe11870@mbuna.arbhome.com.au> Message-ID: <200204081616.g38GG5w12650@pcp742651pcs.reston01.va.comcast.net> > >>> Guido van Rossum wrote > > Note that I was just in time checking in the backward compatibility > > bool(), True and False builtins (returning ints, of course). If > > Anthony is listening, could I add those to 2.1.3 too? This will ease > > backporting 2.3 code to 2.1.3 and 2.2.1. > > I can't decide on whether this should go in 2.1.3 or not. On the > plus side, it's unlikely to break things. On the minus side, it is > breaking the "contract"[*] about what goes into a minor release. > > [*] where "contract" is between the python developers and the people > who install and use it. up until now, we've said 'minor releases are > bugfixes only' - changing this at the last minute before a release > feels a bit wrong, somehow. > > Given the nature of the bug fixes in 2.1.3, I _really_ don't want to > give people _any_ fears that they might have backwards compatability > concerns to deal with. I can't see any way that adding bool/True/False > to 2.1.3 will break things, but it's a perceptions thing, I guess. I > just want to see 2.1, 2.1.1 and 2.1.2 go away :) > > Hm. At this point (well, at the point I got Guido's email), the 2.1.3 > release is pretty much done but for > . make install on creosote. > . rename the file in ftp.python.org (it's called rc0 right now). > . pushing the 'hidden->active' button on sf. > . sending the email announcement. > If I'm going to add this, I'll need to recut the release. In any case, > I'm not going to do this right now (it's late) - I'll look at it first > thing tomorrow and try to decide then. I really _would_ like feedback, > though. I think our current discussion is pointing towards a (slight) loosening up of the rules for micro releases, but I think that it's better not to push this particular fix into 2.1.3 after all. I did push it into 2.2.1 -- we've already added enough tiny features that I think this is acceptable. I think I'd like to sell it as a way to ease backporting 2.3 library modules to 2.2.x, rather than as a way for people to get used to writing bool(x) instead of not not x, and True or False instead of 1 or 0. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 8 17:21:00 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 12:21:00 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Tue, 09 Apr 2002 01:32:11 +1000." <200204081532.g38FWBc12664@mbuna.arbhome.com.au> References: <200204081532.g38FWBc12664@mbuna.arbhome.com.au> Message-ID: <200204081621.g38GL0N12681@pcp742651pcs.reston01.va.comcast.net> > one last thing before I head off to sleep - it seems to me that > one of the groups we really want to make sure we support is the > people doing the development of python. That's without question. The whole discussion is how we can *also* make the rest of the world happy. > If making lots of micro-releases is going to lead to far more triage > work and chasing up bug reports, then that's a bad bad thing. I'm not sure I understand what you're saying here. If you're saying that more micro releases will lead users to report more bugs in them, that's a *good* thing. We *want* users to report where our new code breaks things for them, and we're having a hard time getting them to download the alpha/beta releases.[*] OTOH, if you're worried that we'll try to clear the bugs database before each micro release, forget it -- that will never happen. If anything, PEP 102 should become a lot shorter. [*] Unfortunately, I'm not so sure that changing labels will help much. Few people download alpha or beta releases. Why would they start download micro releases, when we make it clear that those are just as unstable? --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Mon Apr 8 17:23:39 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 8 Apr 2002 12:23:39 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15537.48492.167677.298169@12-248-41-177.client.attbi.com> References: <15537.46049.729997.710611@12-248-41-177.client.attbi.com> <200204081528.g38FSi112614@mbuna.arbhome.com.au> <15537.48492.167677.298169@12-248-41-177.client.attbi.com> Message-ID: <15537.50187.77727.205845@grendel.zope.com> Skip Montanaro writes: > I don't know. I see a lot more version-related checkins than that float by > just before a release. What about the docs for example? The docs generate at least two of these: Doc/Makefile Doc/texinputs/boilerplate.tex The Makefile case probably can't be automated, since it changes to add a "patchlevel" of it's own when the release is done separately from a Python release (such as the 2.2p1 doc release). We've traditionally also "hard-coded" the release date into the source boilerplate.tex, so that formatting the released doc sources doesn't generate a new date. (Switching CVS back to \today afterward generates yet another checkin.) There's probably a better way to handle this -- I don't like all the dance steps we have to do for this either! -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From guido@python.org Mon Apr 8 17:28:39 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 12:28:39 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 11:38:19 EDT." <15537.47467.132247.414879@anthem.wooz.org> References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> <15537.43727.369744.356521@grendel.zope.com> <15537.47467.132247.414879@anthem.wooz.org> Message-ID: <200204081628.g38GSet12746@pcp742651pcs.reston01.va.comcast.net> > I wonder if decoupling more of the standard library isn't a good idea, > although it flies in the face of "batteries included". I think it will be a *bad* thing, unless we can automate it to perfection (more about that below). > One thing I like about XEmacs's model is that I kind of get both for > little effort. I want all the latest packages, but I don't want to > re-install XEmacs to get them. So I use the package manager, which > isn't exactly one-click-updates, but it's pretty simple to use. For most mortals, the XEmacs package manager simply *sucks*. Maybe that is because it's broken out of the box and you have to find the right webpage that tells you how to manually fix some things. But it still sucks. The only thing that *might* work IMO would be something that pulls down the latest (compatible) library modules when you say "make install", or in a post-install script when using RPMs or some such thing. But even then, the people distributing Python on CD with their books lose -- those CDs are there for folks who have a slow or non-existent network connection. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 8 17:30:10 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 12:30:10 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Tue, 09 Apr 2002 01:28:44 +1000." <200204081528.g38FSi112614@mbuna.arbhome.com.au> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> Message-ID: <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> > That tide's only the linux users, and the linux users who care what > kernel they're running. I'm an ex-sysadmin, pretty solid unix geek, > and I nowadays run whatever my distro comes with - as long as it > supports what I want to do, I don't care. I don't think that saying > "it's what linux does" is enough to carry the day by itself. I suspect > that if you want to go with the flow, we should follow the approach that > Microsoft uses to mark their unstable releases. (And I'm _not_ going to > use any one of the many, many cheap shots about unstable MS releases > that I have in my head here - feel free to substitute your own. :) Cheap shots aside, another possibility would be to simply start *every* minor (2.x) release off as unstable, releasing frequent experimental micro (2.x.y) releases as a substitute for alpha/beta releases, and then at some point declare it stable. At that point, the previous stable release (2.(x-1)) becomes deprecated and largely unmaintained (except for backporting some killer bugs -- the mode we seem to be in for 2.1.3 already), 2.x starts issueing micro releases that mostly fix bugs or add very non-controversial features, and 2.(x+1) is started for new development. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 8 17:33:35 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 12:33:35 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 10:55:24 CDT." <15537.48492.167677.298169@12-248-41-177.client.attbi.com> References: <15537.46049.729997.710611@12-248-41-177.client.attbi.com> <200204081528.g38FSi112614@mbuna.arbhome.com.au> <15537.48492.167677.298169@12-248-41-177.client.attbi.com> Message-ID: <200204081633.g38GXZ912787@pcp742651pcs.reston01.va.comcast.net> > I don't know. I see a lot more version-related checkins than that > float by just before a release. What about the docs for example? If we did frequent micro releases, I would recommend doing the Doc, Win and Mac releases on independ schedules. The micro release would only be a source tarball (including whatever state the Doc, Win and Mac subtrees are at the time). Maybe it would be good to do another source micro release just after a Mac or Win micro release was made, but that would be optional. --Guido van Rossum (home page: http://www.python.org/~guido/) From petrilli@amber.org Mon Apr 8 17:31:06 2002 From: petrilli@amber.org (Christopher Petrilli) Date: Mon, 08 Apr 2002 12:31:06 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081607.g38G7iD12531@pcp742651pcs.reston01.va.comcast.net> Message-ID: On 4/8/02 12:07 PM, "Guido van Rossum" wrote: > In Python's case, that's a vanishingly small fraction of the total user base. > Even the alpha/beta releases are downloaded by a disappointingly small number > of people. Many Python users apparently consider themselves early adopters > when they install 2.2 final when it comes out. > This sounds to me like we must strive to make those releases as stable as possible, rather than the reverse. Certainly, when I download something that is termed a release, I think of something that's been tested. Perhaps we simply need to more actively solicit people to participate in more early stages of the process. > Unfortunately, what's frequent for one person appears at glacial speed for > another. Since 2.0 came out, we've new Python releases (counting only the > minor version numbers) every 6-8 months. Apparently the conservative forces > in the community find this too often. > Are the conservative forces truly the majority, or are they simply the most vocal? Often those whose voices are loudest represent the smallest percentage of the population. Perhaps the best thing to do is to take a poll, or something. This isn't great, but it certainly is more indicative than a self-selecting group that complaint about something. WE all know that thanks comes rarely, and only after complaints. A lot of people, myself included, don't bother to read the news groups because of the noise ratio. This selects out those who are "content" with the status quo. > *I* though that PEP 285 did everything humanly possible to make the bool type > backward compatible. But according to the c.l.py crowd it's not enough. The people I care about are not the people who "play" with Python for fun and games, but those who do serious work with it. For example, Red Hat, and their installer work, the guys at the Hubbel Telescope and NASA, LLNL, Zope, others who we don't know about. These are the people with something on the line besides their preferences when the Python core team makes a decision. I don't want to say someone is a second class citizen, but certainly, one must weigh the importance of having a specific user. If we had thousands of individuals clamoring for a new process, it would sway me more than a few vocal people. > Try telling Logajan or Rubin that. :-( There will always be disagreement with the decisions, regardless of what the decision might be. The goal is to please a majority. > I sure hope so, but in practice, even the smallest incompatibility between 2.Y > and 2.(Y+1) seems to trip people up. Most of my "trip ups" have been sloppy coding on my part. The only one that nailed me of any size was the change in various things that now require tuples. It said it in the docs, but hey, I didn't read the docs! ;-) This is one place where a warning would have definitely helped a lot more, but in the end was my mistake. Perhaps Python needs a --pedantic option? :-) > Have multiple versions installed, with binaries named "python2.1, python2.2, > python2.3" and "python" linked to the most popular one. Bleeding edge users > can place a symlink to python2.3 in their personal bin directory if they don't > want to type "python2.3" all the time. Exactly, and something I know dozens of people personally who do. I've even considered removing "python" as a link to anything, except for sloppy code that uses it rather than calling the version it expects. Certainly those who have worked with Java/JVM know the pain of version incompatibilities better than we ever have. >> I will point out that it is trivial to maintain multiple X.Y releases on the >> same machine, and for example, I have 1.5.2, 2.1 and 2.2 on my main >> development machine currently, and simply always make sure to specify the >> Python release in the #! header at the top of the file. This way in the >> future, I can still use it. >> > Apparently few people know how to use this properly. Also, there are those > who want the impossible: they want no old code to break, yet they want to be > able to use bleeding edge packages. The first is solvable by documentation and education, perhaps even by modifying how the installer works so encourage this scenario. Change all the scripts to use the preferred syntax in the #! component, and make sure we reinforce use of the different library releases. As for the later issue, medication is perhaps the only suitable option. >> A panacea of universal happiness is a wonderful thing, but I think we need to >> step back and answer the question of who we are trying to please with these >> proposed changes, whether that group is in fact using Python at any large >> level, or would use Python at any large level, and whether or not we would >> alienate people who have been with the community for a large period of time. >> > This certainly seems closer to resolution that peace in the Middle East. :-) Intransigence is often the primary problem in all conflicts :-) >> As someone commented (and I can't remember who now), this needs to be taken >> in the context of increasing the spread of Python, rather than pleasing a >> potentially vocal minority. If such a change is advantageous to the spread >> of Python and its adoption, then I'm 100% for it. >> > Unfortunately, the vocal minority is claiming that Python's fast-paced > releases are what keeps the spread from going up. Are they right? Who knows? Instinct has served you and the community well so far, and you are the BDFL. I would say take a poll of people who have been involved in the community for an extended period, and we all know who they are. Find out what they think the issue is, and the resolution. We'll call it representative democracy. :-) Chris -- | Christopher Petrilli From mwh@python.net Mon Apr 8 17:34:18 2002 From: mwh@python.net (Michael Hudson) Date: 08 Apr 2002 17:34:18 +0100 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Guido van Rossum's message of "Mon, 08 Apr 2002 12:21:00 -0400" References: <200204081532.g38FWBc12664@mbuna.arbhome.com.au> <200204081621.g38GL0N12681@pcp742651pcs.reston01.va.comcast.net> Message-ID: <2m8z7y6tf9.fsf@starship.python.net> Guido van Rossum writes: > If anything, PEP 102 should become a lot shorter. Not buggering around releasing files on sf would help here. Why do we do this again? (Particularly now even I can update files on www.python.org...) Cheers, M. -- 48. The best book on programming for the layman is "Alice in Wonderland"; but that's because it's the best book on anything for the layman. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html From akuchlin@mems-exchange.org Mon Apr 8 17:39:03 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Mon, 8 Apr 2002 12:39:03 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020408163903.GA2216@crystal.mems-exchange.org> On Mon, Apr 08, 2002 at 12:30:10PM -0400, Guido van Rossum wrote: >Cheap shots aside, another possibility would be to simply start >*every* minor (2.x) release off as unstable, releasing frequent >experimental micro (2.x.y) releases as a substitute for alpha/beta >releases, and then at some point declare it stable. I like this approach because it matches what the community seems to actually be doing: the latest release is "experimental" until told otherwise. (Though who decides when it stops being experimental? Everyone here on python-dev thought 2.2 was OK, and I use 2.2 every day without pain, so I don't really know what people are complaining about.) --amk (www.amk.ca) ... like a bull in a barber shop. -- The Doctor, in "Time and the Rani" From akuchlin@mems-exchange.org Mon Apr 8 18:13:16 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Mon, 8 Apr 2002 13:13:16 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15537.47467.132247.414879@anthem.wooz.org> References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> <15537.43727.369744.356521@grendel.zope.com> <15537.47467.132247.414879@anthem.wooz.org> Message-ID: <20020408171316.GC2216@crystal.mems-exchange.org> On Mon, Apr 08, 2002 at 11:38:19AM -0400, Barry A. Warsaw wrote: >I wonder if decoupling more of the standard library isn't a good idea, >although it flies in the face of "batteries included". One thing I I'd like this approach, though perhaps the "Batteries Included" distribution rather than the standard library should be handled like this. It would also require finalizing a catalog implementation (at last!), An open question is how to avoid the CPAN trap of having your modules require you to upgrade your Python installation. Perhaps each module version would specify the Python version it's compatible with. Module version compat. with 0.0.1 Py2.1 0.0.2 Py2.1 0.0.5 Py2.2 So when you're on Python 2.1, you can only get upgraded to 0.0.2. The problem is, if the developer of version 0.0.5 didn't realize that it was incompatible with 2.1, you'll get upgraded to it and your system will be broken. (Take package management far enough and you begin to reinvent Debian's APT. I wonder if we could write a dpkg-like tool in Python and then just use APT on top of that?) --amk (www.amk.ca) Nothing I've ever written has reached 1.0. -- Greg Ward at IPC7, on using small version numbers From guido@python.org Mon Apr 8 18:18:37 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 13:18:37 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "08 Apr 2002 17:34:18 BST." <2m8z7y6tf9.fsf@starship.python.net> References: <200204081532.g38FWBc12664@mbuna.arbhome.com.au> <200204081621.g38GL0N12681@pcp742651pcs.reston01.va.comcast.net> <2m8z7y6tf9.fsf@starship.python.net> Message-ID: <200204081718.g38HIbA13032@pcp742651pcs.reston01.va.comcast.net> > > If anything, PEP 102 should become a lot shorter. > > Not buggering around releasing files on sf would help here. Why do we > do this again? (Particularly now even I can update files on > www.python.org...) We thought it would be good PR. I kind of doubt its effectiveness. Let's stop after 2.2. (Or do you want to stop before 2.2.1? Fine with me, too.) --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Mon Apr 8 18:20:31 2002 From: fdrake@acm.org (Fred Drake) Date: Mon, 08 Apr 2002 13:20:31 -0400 Subject: [Python-Dev] Releasing files via SF In-Reply-To: <200204081718.g38HIbA13032@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Regarding releasing files on SF.] On Mon, 08 Apr 2002 13:18:37 -0400 Guido van Rossum wrote: > We thought it would be good PR. I kind of doubt its > effectiveness. Let's stop after 2.2. (Or do you want to > stop before 2.2.1? Fine with me, too.) I'm incredibly happy not to do SF releases of the docs! ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From guido@python.org Mon Apr 8 18:24:24 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 13:24:24 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 12:39:03 EDT." <20020408163903.GA2216@crystal.mems-exchange.org> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> Message-ID: <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> [me] > >Cheap shots aside, another possibility would be to simply start > >*every* minor (2.x) release off as unstable, releasing frequent > >experimental micro (2.x.y) releases as a substitute for alpha/beta > >releases, and then at some point declare it stable. [AMK] > I like this approach because it matches what the community seems to > actually be doing: the latest release is "experimental" until told > otherwise. (Though who decides when it stops being experimental? > Everyone here on python-dev thought 2.2 was OK, and I use 2.2 every > day without pain, so I don't really know what people are complaining > about.) Maybe different user sub-communities can keep their own table of which versions they consider sufficiently stable to use. E.g. I can see Zope Corp declaring Python 2.1.3 the holy grail for Zope 2.5 and 2.6, but the Zope 3 developers will probably want to start using 2.3 ASAP. All the Python developers need to do is decide on a point where they won't allow incompatible changes to new features between micro releases. Currently that point is the release of the "final" release of a minor version (e.g. 2.2). I can see that point shifting to a later micro release for future minor releases, so that e.g. 2.3, 2.3.1 up to 2.3.7 may contain incompatible features, but 2.3.8 and later must be compatible with 2.3.7. --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Mon Apr 8 18:22:48 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Apr 2002 13:22:48 -0400 Subject: [Python-Dev] Re: Stability and change References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> <15537.43727.369744.356521@grendel.zope.com> <15537.47467.132247.414879@anthem.wooz.org> <200204081628.g38GSet12746@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15537.53736.695434.259081@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> For most mortals, the XEmacs package manager simply *sucks*. GvR> Maybe that is because it's broken out of the box and you have GvR> to find the right webpage that tells you how to manually fix GvR> some things. But it still sucks. I agree totally, but that's a quality of implementation problem. Once it works, it's (almost) great. But you're right, it would have been nearly completely automatic, and bug free out of the box. I guess it's no surprise we've talked about this for years but it still hasn't materialized. -Barry From aleax@aleax.it Mon Apr 8 18:24:45 2002 From: aleax@aleax.it (Alex Martelli) Date: Mon, 8 Apr 2002 19:24:45 +0200 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: <20020408163903.GA2216@crystal.mems-exchange.org> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> Message-ID: On Monday 08 April 2002 06:39 pm, Andrew Kuchling wrote: ... > otherwise. (Though who decides when it stops being experimental? > Everyone here on python-dev thought 2.2 was OK, and I use 2.2 every > day without pain, so I don't really know what people are complaining 2.2 is a quite-stable piece of software, per se. When you ask if a piece of software is stable, out of context, you normally think of crashes and other sundry bugs. In that sense, the complaints are not well-founded, in my opinion. The Python language was not 'stable' between 2.1.* and 2.2 : some previously correct programs stopped working. That's according to plan. BUT -- that plan is a problem for people who need their code, if correct, to keep working for longer than the 6 months between minor releases. Right now we're telling them to stick with (e.g.) 2.1 and just hope that somebody is interested enough to backport bug fixes, enhancements that would break nothing (such as new library modules) or "nothing important" (a far harder call!-), and so on. I do not think this is optimal. On one hand it may over-constrain Guido from release to release. He can't put a feature in 2.3 without thereby befoming unable to take it away again, or redo it in some incompatible way, should it prove imperfect. If 2.3 was an admittedly experimental release, he COULD choose to take more risks in this sense! He may afford a little bit less stability in the design of languages he releases (not necessarily in terms of "it crashes every minute" -- that's ANOTHER sense of "stable"... the one I was mentioning 2 para's ago, and the one I think you're asking about). On the other hand, software dev't managers don't like the choice between an "old, obsolete, probably not of interest any more" piece of software, and one that's "so brand-new it's still churning AND may break your existing correct code". It seems to me that trying to reasonably meet all needs within one "track" of development is doomed to fail, while "two tracks" would afford a better chance. The "stable" track would focus on not breaking existing correct programs; the "experimental" track would focus on enhancements with more freedom (including freedom to take some language-design risks). Thus, "who decides" would have to be GvR -- but it woulnd't most likely be twice a year that the stable track gets to a head and needs a new baseline (stabilized from a line of experimental track releases). The lucky accident of 1.5.2's long stability, which attracted so many to Python, would ideally be reproduced by design... or so one can hope. Alex From tim.one@comcast.net Mon Apr 8 18:24:26 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 08 Apr 2002 13:24:26 -0400 Subject: [Python-Dev] 2.1.3 versus Windows In-Reply-To: <200204081442.g38Egpe11870@mbuna.arbhome.com.au> Message-ID: Is PLabs releasing a Windows installer for 2.1.3? If so, I should at least backport the zlib "security hole" update. From aleax@aleax.it Mon Apr 8 18:27:36 2002 From: aleax@aleax.it (Alex Martelli) Date: Mon, 8 Apr 2002 19:27:36 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> Message-ID: On Monday 08 April 2002 07:24 pm, Guido van Rossum wrote: ... > All the Python developers need to do is decide on a point where they > won't allow incompatible changes to new features between micro > releases. Currently that point is the release of the "final" release > of a minor version (e.g. 2.2). I can see that point shifting to a > later micro release for future minor releases, so that e.g. 2.3, 2.3.1 > up to 2.3.7 may contain incompatible features, but 2.3.8 and later > must be compatible with 2.3.7. It seems to me that the "stabilization" point SHOULD be more clearly marked -- get the message out "this IS stable and reliable, anything written for this release will not break in future releases of the same major.minor" in a very clear way. It seems to me that duplicating (in your example) 2.3.8 to 2.4.0 (and using 2.5.0 as the new baseline for further experimentation) would be a very clear signal in this sense. Alex From fdrake@acm.org Mon Apr 8 18:28:28 2002 From: fdrake@acm.org (Fred Drake) Date: Mon, 08 Apr 2002 13:28:28 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081621.g38GL0N12681@pcp742651pcs.reston01.va.comcast.net> Message-ID: On Mon, 08 Apr 2002 12:21:00 -0400 Guido van Rossum wrote: > I'm not sure I understand what you're saying here. If > you're saying > that more micro releases will lead users to report more > bugs in them, If they report bugs sooner, that is good. My first take on the question was that we'd probably see more duplicate reports for things that didn't get fixed in the next point release. It's not just a matter of fixing more bugs or when we fix them, it's how much overhead we have doing triage in the issue tracker. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From guido@python.org Mon Apr 8 18:34:58 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 13:34:58 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 13:13:16 EDT." <20020408171316.GC2216@crystal.mems-exchange.org> References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> <15537.43727.369744.356521@grendel.zope.com> <15537.47467.132247.414879@anthem.wooz.org> <20020408171316.GC2216@crystal.mems-exchange.org> Message-ID: <200204081734.g38HYwb13124@pcp742651pcs.reston01.va.comcast.net> [Barry] > >I wonder if decoupling more of the standard library isn't a good idea, > >although it flies in the face of "batteries included". One thing I [AMK] > I'd like this approach, though perhaps the "Batteries Included" > distribution rather than the standard library should be handled like > this. For a batteries included release, this would be great. > It would also require finalizing a catalog implementation (at > last!) I believe Kapil is working on it; he's presenting a paper at OSCON2002. > An open question is how to avoid the CPAN trap of having your modules > require you to upgrade your Python installation. Perhaps each module > version would specify the Python version it's compatible with. > > Module version compat. with > 0.0.1 Py2.1 > 0.0.2 Py2.1 > 0.0.5 Py2.2 Ideally it should list a range of versions (or a set?). Of course, the upper end of the range may need to be adjusted when new releases come out -- nobody knows whether Module 0.0.5 will be compatible with Py2.3 until the latter is released. > So when you're on Python 2.1, you can only get upgraded to 0.0.2. The > problem is, if the developer of version 0.0.5 didn't realize that it > was incompatible with 2.1, you'll get upgraded to it and your system > will be broken. > > (Take package management far enough and you begin to reinvent Debian's > APT. I wonder if we could write a dpkg-like tool in Python and then > just use APT on top of that?) I think version dependency management is subject to the Logajan paradox. :-( --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 8 18:35:50 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 13:35:50 -0400 Subject: [Python-Dev] Releasing files via SF In-Reply-To: Your message of "Mon, 08 Apr 2002 13:20:31 EDT." References: Message-ID: <200204081735.g38HZpT13146@pcp742651pcs.reston01.va.comcast.net> > I'm incredibly happy not to do SF releases of the docs! ;-) OK, go for it. Let's drop all SF file releases as of now. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 8 18:37:10 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 13:37:10 -0400 Subject: [Python-Dev] Re: 2.1.3 versus Windows In-Reply-To: Your message of "Mon, 08 Apr 2002 13:24:26 EDT." References: Message-ID: <200204081737.g38HbAU13166@pcp742651pcs.reston01.va.comcast.net> > Is PLabs releasing a Windows installer for 2.1.3? > > If so, I should at least backport the zlib "security hole" update. We just decided "yes" on xchat. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Mon Apr 8 18:36:06 2002 From: skip@pobox.com (Skip Montanaro) Date: Mon, 8 Apr 2002 12:36:06 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081444.g38Ei6V12247@pcp742651pcs.reston01.va.comcast.net> References: <15537.15490.631372.551394@12-248-41-177.client.attbi.com> <200204080652.g386qBS06363@mbuna.arbhome.com.au> <15537.41843.997924.386742@12-248-41-177.client.attbi.com> <200204081444.g38Ei6V12247@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15537.54534.570764.806846@12-248-41-177.client.attbi.com> >>>>> "Guido" == Guido van Rossum writes: >> I don't know if I understand what you're getting at here. I think >> that 2.x.a -> 2.x.b should be relatively stable, certainly if x is >> odd. Less so if x is even, but most of the time not dramatically so. Guido> Ehm, haven't you got that backwards? I did "uname -a" and my Guido> kernel version is 2.4.9, which would suggest that even minor Guido> numbers are stable for Linux. And you just agreed that we should Guido> do the same for Python. 2.4.x is the stable Linux kernel. People had been talking about Python 2.1.x as the "Garth" (we fear change) series. I was simply going along with that theme. >> (One way to accomplish this would be to have a standard patch file >> whose version numbers are twiddled, probably by a script, and which >> is then applied from the top of the source tree.) Guido> You'd still have to watch it though. Yes, but I suspect generating a new version of that diff file shouldn't be too difficult on the occasions where some new location containing a version number pops up. Skip From aleax@aleax.it Mon Apr 8 18:35:18 2002 From: aleax@aleax.it (Alex Martelli) Date: Mon, 8 Apr 2002 19:35:18 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: Message-ID: On Monday 08 April 2002 06:31 pm, Christopher Petrilli wrote: ... > Are the conservative forces truly the majority, or are they simply the most > vocal? Often those whose voices are loudest represent the smallest > percentage of the population. Perhaps the best thing to do is to take a > poll, or something. This isn't great, but it certainly is more indicative > than a self-selecting group that complaint about something. WE all know If you poll people who are using a language that's "churning very often", you WILL get a self-selecting group biased towards being happy with that rate of churning. What I think is that Python's user base could be expanded if (without making that group sad: keeping an experimental track with frequent churning around!!!) we could have a parallel, stabler track. > certainly, one must weigh the importance of having a specific user. If we > had thousands of individuals clamoring for a new process, it would sway me > more than a few vocal people. How would we get the people who won't use Python because they don't perceive it as stable enough for their needs to "clamor" for changes to its release tracks?-) I can't "sell" Python to those people until I pass that roadblock. > > Try telling Logajan or Rubin that. :-( > > There will always be disagreement with the decisions, regardless of what > the decision might be. The goal is to please a majority. A majority of EXISTING users, or a majority of those we COULD have with more language stability?-) We can please BOTH... by doing dual tracks... At least, it worked wonderfully for Linux. Which is where I started this whole thread of discussion... Alex From guido@python.org Mon Apr 8 18:38:57 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 13:38:57 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 19:27:36 +0200." References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204081738.g38HcvF13190@pcp742651pcs.reston01.va.comcast.net> [me] > > All the Python developers need to do is decide on a point where they > > won't allow incompatible changes to new features between micro > > releases. Currently that point is the release of the "final" release > > of a minor version (e.g. 2.2). I can see that point shifting to a > > later micro release for future minor releases, so that e.g. 2.3, 2.3.1 > > up to 2.3.7 may contain incompatible features, but 2.3.8 and later > > must be compatible with 2.3.7. [Alex] > It seems to me that the "stabilization" point SHOULD be more > clearly marked -- get the message out "this IS stable and reliable, > anything written for this release will not break in future releases > of the same major.minor" in a very clear way. Yes. > It seems to me that duplicating (in your example) 2.3.8 to 2.4.0 > (and using 2.5.0 as the new baseline for further experimentation) > would be a very clear signal in this sense. *If* the community likes the even/odd distinction enough. I've heard mixed feelings. --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Mon Apr 8 18:36:10 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Apr 2002 13:36:10 -0400 Subject: [Python-Dev] Re: Stability and change References: <200204081532.g38FWBc12664@mbuna.arbhome.com.au> <200204081621.g38GL0N12681@pcp742651pcs.reston01.va.comcast.net> <2m8z7y6tf9.fsf@starship.python.net> <200204081718.g38HIbA13032@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15537.54538.681864.373070@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: >> If anything, PEP 102 should become a lot shorter. >> Not buggering around releasing files on sf would help here. >> Why do we do this again? (Particularly now even I can update >> files on www.python.org...) GvR> We thought it would be good PR. I kind of doubt its GvR> effectiveness. Let's stop after 2.2. (Or do you want to GvR> stop before 2.2.1? Fine with me, too.) I'd have no problem with this, /if/ you removed the old files from SF. The worst you can do is leave old files laying around while we're uploading all the new versions to python.org. -Barry From guido@python.org Mon Apr 8 18:40:58 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 13:40:58 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 13:28:28 EDT." References: Message-ID: <200204081740.g38Hewj13211@pcp742651pcs.reston01.va.comcast.net> [me] > > I'm not sure I understand what you're saying here. If you're > > saying that more micro releases will lead users to report more > > bugs in them, [Fred] > If they report bugs sooner, that is good. My first take on > the question was that we'd probably see more duplicate > reports for things that didn't get fixed in the next point > release. It's not just a matter of fixing more bugs or when > we fix them, it's how much overhead we have doing triage in > the issue tracker. I think dups are usually spotted pretty easily. We could add a standard response clarifying this. No need to point people to the specific SF # they are duplicating. --Guido van Rossum (home page: http://www.python.org/~guido/) From aleax@aleax.it Mon Apr 8 18:41:27 2002 From: aleax@aleax.it (Alex Martelli) Date: Mon, 8 Apr 2002 19:41:27 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081738.g38HcvF13190@pcp742651pcs.reston01.va.comcast.net> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081738.g38HcvF13190@pcp742651pcs.reston01.va.comcast.net> Message-ID: On Monday 08 April 2002 07:38 pm, Guido van Rossum wrote: ... > > It seems to me that duplicating (in your example) 2.3.8 to 2.4.0 > > (and using 2.5.0 as the new baseline for further experimentation) > > would be a very clear signal in this sense. > > *If* the community likes the even/odd distinction enough. I've heard > mixed feelings. I'm neutral on the odd/even distinction -- all it has for it is that a number of people are used to it from Linux. But SOMEthing more remarkable that "up to 2.3.N experimental, 2.3.(N+1) and on stable" is needed -- a different name, a different major release, whatever. If we can come up with something better than parity of minor release number, I'll personally cheer... I just can't think of anything better right now. Comparison of microrelease with some arbitrary threshold changing by minor.major is worse, though -- not a clear signal at all. Alex From barry@zope.com Mon Apr 8 18:45:44 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Apr 2002 13:45:44 -0400 Subject: [Python-Dev] Re: Stability and change References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15537.55112.191870.654238@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> All the Python developers need to do is decide on a point GvR> where they won't allow incompatible changes to new features GvR> between micro releases. Currently that point is the release GvR> of the "final" release of a minor version (e.g. 2.2). I can GvR> see that point shifting to a later micro release for future GvR> minor releases, so that e.g. 2.3, 2.3.1 up to 2.3.7 may GvR> contain incompatible features, but 2.3.8 and later must be GvR> compatible with 2.3.7. Maybe I misunderstand, but it makes me really nervous that micro releases would have internal incompatibilities (I'm not talking about those related to fixed bugs in earlier micros). One that I've noticed is that many, many people simply do not understand the concept of a micro release. I can't tell you how many questions I get that ask about Mailman 2.08 (sic). We saw some of this when Python 1.5.2 ("Python 1.52") was out. I'd dread having to explain to people "this is compatible with Python Python 2.4.2 but not anything earlier than 2.4.1 unless you're on the 2.3.x branch and then 2.3.7 is okay, but not 2.3.6 or earlier." -Barry From guido@python.org Mon Apr 8 18:55:32 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 13:55:32 -0400 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: Your message of "Mon, 08 Apr 2002 19:24:45 +0200." References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> Message-ID: <200204081755.g38HtXt13343@pcp742651pcs.reston01.va.comcast.net> > The Python language was not 'stable' between 2.1.* and 2.2 : some > previously correct programs stopped working. That's according to > plan. BUT -- that plan is a problem for people who need their code, > if correct, to keep working for longer than the 6 months between > minor releases. Right now we're telling them to stick with (e.g.) > 2.1 and just hope that somebody is interested enough to backport > bug fixes, enhancements that would break nothing (such as new > library modules) or "nothing important" (a far harder call!-), and > so on. I do not think this is optimal. Maybe not, but it works pretty well in the one case I'm familiar with, Zope. We recommend using Python 2.1.2 (and as of tomorrow, 2.1.3) for Zope 2.4 and 2.5, and probably also for Zope 2.6 once it's out. > On one hand it may over-constrain Guido from release to release. > He can't put a feature in 2.3 without thereby befoming unable to take it > away again, or redo it in some incompatible way, should it prove > imperfect. If 2.3 was an admittedly experimental release, he COULD > choose to take more risks in this sense! Maybe, but in practice, any major new feature (say, of the level of list comprehensions or augmented assignment) takes too much effort to add lightly, and once it's there, it's too enticing not to start using in the standard library -- and then I can't take it away at all. At best, I'll be able to tweak it a bit (which is better than now, admittedly). > He may afford a little bit less stability in the design of languages > he releases (not necessarily in terms of "it crashes every minute" > -- that's ANOTHER sense of "stable"... the one I was mentioning 2 > para's ago, and the one I think you're asking about). > > On the other hand, software dev't managers don't like the choice > between an "old, obsolete, probably not of interest any more" piece > of software, and one that's "so brand-new it's still churning AND > may break your existing correct code". But that's still the choice they get, albeit phrased differently. > It seems to me that trying to reasonably meet all needs within one > "track" of development is doomed to fail, while "two tracks" would > afford a better chance. The "stable" track would focus on not > breaking existing correct programs; the "experimental" track would > focus on enhancements with more freedom (including freedom to > take some language-design risks). I'm not sure the stable track would differ in practice from what we're already doing with 2.1.3 and 2.2.1. > Thus, "who decides" would have to be GvR -- but it woulnd't most > likely be twice a year that the stable track gets to a head and > needs a new baseline (stabilized from a line of experimental > track releases). The lucky accident of 1.5.2's long stability, which > attracted so many to Python, would ideally be reproduced by > design... or so one can hope. I don't know that 1.5.2's stability attracted people. It was probably more its feature-fullness, plus the fact that around the same time lots of books etc. started to appear. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 8 18:55:58 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 13:55:58 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 13:36:10 EDT." <15537.54538.681864.373070@anthem.wooz.org> References: <200204081532.g38FWBc12664@mbuna.arbhome.com.au> <200204081621.g38GL0N12681@pcp742651pcs.reston01.va.comcast.net> <2m8z7y6tf9.fsf@starship.python.net> <200204081718.g38HIbA13032@pcp742651pcs.reston01.va.comcast.net> <15537.54538.681864.373070@anthem.wooz.org> Message-ID: <200204081755.g38Htwr13354@pcp742651pcs.reston01.va.comcast.net> > GvR> We thought it would be good PR. I kind of doubt its > GvR> effectiveness. Let's stop after 2.2. (Or do you want to > GvR> stop before 2.2.1? Fine with me, too.) > > I'd have no problem with this, /if/ you removed the old files from > SF. The worst you can do is leave old files laying around while we're > uploading all the new versions to python.org. Nothing's easier than that. --Guido van Rossum (home page: http://www.python.org/~guido/) From petrilli@amber.org Mon Apr 8 18:52:56 2002 From: petrilli@amber.org (Christopher Petrilli) Date: Mon, 08 Apr 2002 13:52:56 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <20020408174700.EED055DC05@trump.amber.org> Message-ID: On 4/8/02 1:35 PM, "Alex Martelli" wrote: > How would we get the people who won't use Python because they > don't perceive it as stable enough for their needs to "clamor" for > changes to its release tracks?-) I can't "sell" Python to those people > until I pass that roadblock. I think those people for whom any backward incompatible change is anathema must assume upon themselves the burden of maintenance of the code base. I have customers who are still on Solaris 2.7, but they know they can't buy certain hardware and have it work. Part of the burden of buying that new hardware is upgrading their software to meet the requirements. If you want something, sometimes you must make a sacrifice. Obviously if someone's software is totally functional on 2.1, then they are moving to 2.2 for one of a couple reasons: 1) Fun 2) They want something new In the first case, I'm not terribly concerned with their pain. In the second, it is reasonable to expect them to make a sacrifice in exchange for the benefit. It's a simple cost-benefit analysis. I want feature X, but to get feature X, will require me to invest Y amount of time. Do the two offset each other? > A majority of EXISTING users, or a majority of those we COULD have > with more language stability?-) > > We can please BOTH... by doing dual tracks... I'd argue that there is a high percentage of users for whom nothing will ever make them happy, these are the fickle users. They are the ones who have 12 different Linux distributions installed, sweat as to when their 8 year old ISA Ethernet board will be supported, and altogether find more use in whining than in doing. These are the people who complain because it makes them feel important. Usually they're confined to talking head shows on TV, but often they spill into other venues, like UseNet. All changes should be backed up by rational justifications, not just "we could have more users". To take a business approach, if we acquire a sudden jump of 10% in the user community, but we find a 50% annual churn rate in that new boost, have we in-fact garnered anything? Or would be better targeting a 5% growth where we can have a 10% churn. In the end, I'd like to see something more concrete than speculation if we're going to change behavior. I'm not opposed to a new model, I just want to understand what it gains in more concrete terms. Speculation as to what the user wants gets you Microsoft Bob. Chris From guido@python.org Mon Apr 8 18:57:31 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 13:57:31 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 19:41:27 +0200." References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081738.g38HcvF13190@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204081757.g38HvVs13378@pcp742651pcs.reston01.va.comcast.net> > I'm neutral on the odd/even distinction -- all it has for it is that a > number of people are used to it from Linux. But SOMEthing more > remarkable that "up to 2.3.N experimental, 2.3.(N+1) and on > stable" is needed -- a different name, a different major release, > whatever. If we can come up with something better than parity > of minor release number, I'll personally cheer... I just can't think > of anything better right now. Comparison of microrelease with > some arbitrary threshold changing by minor.major is worse, though -- > not a clear signal at all. We could jump the micro release number to 100. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 8 19:02:07 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 14:02:07 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 13:45:44 EDT." <15537.55112.191870.654238@anthem.wooz.org> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> Message-ID: <200204081802.g38I27E13430@pcp742651pcs.reston01.va.comcast.net> > Maybe I misunderstand, but it makes me really nervous that micro > releases would have internal incompatibilities (I'm not talking about > those related to fixed bugs in earlier micros). Yes, you would, given that you're the most fear-changing PythonLabber. My proposal was to allow this *only* as long as the corresponding minor release was still clearly labeled as EXPERIMENTAL. > One that I've noticed is that many, many people simply do not > understand the concept of a micro release. I can't tell you how many > questions I get that ask about Mailman 2.08 (sic). We saw some of > this when Python 1.5.2 ("Python 1.52") was out. Well, as long as they only suppress the punctiuation, it's OK. > I'd dread having to explain to people "this is compatible with > Python Python 2.4.2 but not anything earlier than 2.4.1 unless > you're on the 2.3.x branch and then 2.3.7 is okay, but not 2.3.6 or > earlier." Yes, *you* would have to say that, since you like to release software that is on the absolute bleeding edge of the standard libraries or language features. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From akuchlin@mems-exchange.org Mon Apr 8 19:00:43 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Mon, 8 Apr 2002 14:00:43 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: Message-ID: <20020408180043.GA2726@crystal.mems-exchange.org> On Mon, Apr 08, 2002 at 07:35:18PM +0200, Alex Martelli wrote: >At least, it worked wonderfully for Linux. Which is where I started >this whole thread of discussion... It's not clear that it has, given that the Linux development stream is now divided into Linus's 2.5 tree, the 2.5dj tree (Dave Jones), the stable 2.4 tree, the 2.4ac tree (Alan Cox), the 2.4aa tree (Andrea Arcangeli). The only reason this works is that most users never come in contact with these trees; the maintainer of their distribution picks one, and perhaps cherry-picks bits from another tree. The Python analogy would be to ActiveState, or to the numeric user community releasing a Numeric Python distribution. --amk (www.amk.ca) "Victoria wore [that dress]. She travelled with me for a time." "Well, as long as Albert didn't wear it." -- The Doctor and Sarah, in "The Pyramids of Mars" From guido@python.org Mon Apr 8 19:09:14 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 14:09:14 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 12:31:06 EDT." References: Message-ID: <200204081809.g38I9EY13505@pcp742651pcs.reston01.va.comcast.net> > Are the conservative forces truly the majority, or are they simply > the most vocal? I'd be very surprised if they weren't in the majority. It's pretty much what the word conservative *means*, right? --Guido van Rossum (home page: http://www.python.org/~guido/) From paul@svensson.org Mon Apr 8 19:13:51 2002 From: paul@svensson.org (Paul Svensson) Date: Mon, 8 Apr 2002 14:13:51 -0400 (EDT) Subject: [Python-Dev] Re: Stability and change In-Reply-To: Message-ID: On Mon, 8 Apr 2002, Alex Martelli wrote: >On Monday 08 April 2002 07:38 pm, Guido van Rossum wrote: > ... >> > It seems to me that duplicating (in your example) 2.3.8 to 2.4.0 >> > (and using 2.5.0 as the new baseline for further experimentation) >> > would be a very clear signal in this sense. >> >> *If* the community likes the even/odd distinction enough. I've heard >> mixed feelings. > >I'm neutral on the odd/even distinction -- all it has for it is that a >number of people are used to it from Linux. But SOMEthing more >remarkable that "up to 2.3.N experimental, 2.3.(N+1) and on >stable" is needed -- a different name, a different major release, >whatever. If we can come up with something better than parity >of minor release number, I'll personally cheer... I just can't think >of anything better right now. Comparison of microrelease with >some arbitrary threshold changing by minor.major is worse, though -- >not a clear signal at all. Why not take advantage of the "well known fact" that any software release ending in ".0" is not to be trusted ? Make X.Y.0.N the experimental branch, settling down to X.Y.1 as the final stable release, following up with X.Y.Z as bugfix-only releases, and the next round of experimentals continuing from X.(Y+1).0.0. This is almost as simple as the "just call it stable when it is" proposal, while still being even more obvious than the odd/even approach. /Paul From guido@python.org Mon Apr 8 19:18:22 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 14:18:22 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 10:20:04 +0200." <0204081020040G.05787@arthur> References: <02040710094202.05787@arthur> <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <0204081020040G.05787@arthur> Message-ID: <200204081818.g38IIMe13541@pcp742651pcs.reston01.va.comcast.net> > You know PythonLabs's missions and goals better than I do. If such > goals weigh "making a new and better Python" far more highly than > helping spread and preserve the use of Python, then you may well be > right that the extra effort for (e.g.) making email part of the > standard distribution vs the current "maintaining a separate distro" > would be a bad allocation of scarce resources. Actually, in the minds of the people who pay my salary, "making a new and better Python" is one of the best ways to "help spread and preserve the use of Python". > Something requiring much effort is not necessarily "obviously out": > it depends on how important you judge the results of those effort > versus the results you could have by spending the effort elsewhere. I very muich doubt that the corporate users who are currently worried about the fast pace of change are interested in any particular feature, and I don't think that any backported feature is going to make that previous version more popular amongst managers with decision power. The question in front of them is, "should we use Python or not", not "should we use Python 2.1". To answer that question, they want to perceive "Python" as stable, not "Python 2.1". --Guido van Rossum (home page: http://www.python.org/~guido/) From pedroni@inf.ethz.ch Mon Apr 8 19:15:04 2002 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Mon, 8 Apr 2002 20:15:04 +0200 Subject: [Python-Dev] "Unstable" is an ambiguous word... References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081755.g38HtXt13343@pcp742651pcs.reston01.va.comcast.net> Message-ID: <016801c1df29$4eab8000$6d94fea9@newmexico> my 2cts. I think Zope is a bad example, because Zope 2.x is adding features while being developed against 2.1. but Zope 3 is a total rewrite against 2.2 technology. I have the impression that Python is still too young, not strict, and too much reflective and people like to code against the grey area of what I would call implementation details and to avoid reading the docs, that developing (increasing features) only against 2.X and jump to 2.Y (with Y>X) when it becomes "stable", is a bad unpleasant move if the 2.X line will not be here for eternity (and it won't). Unless the whole thing becomes a real (and painful) exercise, in back-porting warnings, and designing transition paths for things that at first sight nobody would have thought they had influence on someone or something. regards. From guido@python.org Mon Apr 8 19:25:51 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 14:25:51 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 01:49:29 EDT." <15537.12137.771457.296841@anthem.wooz.org> References: <200204070709.g3779aJ32707@pcp742651pcs.reston01.va.comcast.net> <15537.12137.771457.296841@anthem.wooz.org> Message-ID: <200204081825.g38IPpp13582@pcp742651pcs.reston01.va.comcast.net> > Note that Jim often brings up his opinion that the /only/ way to > guarantee Zope Z.Y.Z works on Python X.Y.Z is to distribute Python > with Zope. But I don't think that's what we do any more. We just tell them to use 2.1.2. > (This harkens back to JWZ's much earlier opinion that > Netscape could never ship using dynamic libraries, because you really > can't trust anything about the end user's system). We're really > touching on much deeper flaws in software development, with no good > answers, IMO. I've started to call it the Logajan paradox: he wants his cake (zero backwards incompatibilities) and eat it too (releases with new features). It can't be done. --Guido van Rossum (home page: http://www.python.org/~guido/) From nas@python.ca Mon Apr 8 19:30:14 2002 From: nas@python.ca (Neil Schemenauer) Date: Mon, 8 Apr 2002 11:30:14 -0700 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <20020408180043.GA2726@crystal.mems-exchange.org>; from akuchlin@mems-exchange.org on Mon, Apr 08, 2002 at 02:00:43PM -0400 References: <20020408180043.GA2726@crystal.mems-exchange.org> Message-ID: <20020408113014.A24764@glacier.arctrix.com> Andrew Kuchling wrote: > On Mon, Apr 08, 2002 at 07:35:18PM +0200, Alex Martelli wrote: > >At least, it worked wonderfully for Linux. Which is where I started > >this whole thread of discussion... > > It's not clear that it has [...] I agree. Linux is not a good role model. I think FreeBSD has done a much better job with their -STABLE and -CURRENT branch naming. Neil From guido@python.org Mon Apr 8 19:36:21 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 14:36:21 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 14:13:51 EDT." References: Message-ID: <200204081836.g38IaQr13685@pcp742651pcs.reston01.va.comcast.net> > Make X.Y.0.N the experimental branch, > settling down to X.Y.1 as the final stable release, > following up with X.Y.Z as bugfix-only releases, > and the next round of experimentals continuing from X.(Y+1).0.0. This is attractive, if we can make sure that everything settles between 2.x.0 and 2.x.1. That leaves us only alpha and beta releases of 2.x.1 to experiment. Could be a long string of betas... --Guido van Rossum (home page: http://www.python.org/~guido/) From aleax@aleax.it Mon Apr 8 19:37:29 2002 From: aleax@aleax.it (Alex Martelli) Date: Mon, 8 Apr 2002 20:37:29 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081818.g38IIMe13541@pcp742651pcs.reston01.va.comcast.net> References: <0204081020040G.05787@arthur> <200204081818.g38IIMe13541@pcp742651pcs.reston01.va.comcast.net> Message-ID: <02040820372900.14931@arthur> On Monday 08 April 2002 20:18, Guido van Rossum wrote: > > You know PythonLabs's missions and goals better than I do. If such > > goals weigh "making a new and better Python" far more highly than > > helping spread and preserve the use of Python, then you may well be > > right that the extra effort for (e.g.) making email part of the > > standard distribution vs the current "maintaining a separate distro" > > would be a bad allocation of scarce resources. > > Actually, in the minds of the people who pay my salary, "making a new > and better Python" is one of the best ways to "help spread and > preserve the use of Python". Surely having a better Python (by definition of 'better') is in the long run important. I'm not sure of how that importance compares with perceived stability in determining how widespread Python becomes. Judging by languages such as C or C++, stability seems paramount; yet Java churned a lot and still managed to spread a lot too (not without a lot of help from high-$$$ marketing efforts, though). > > Something requiring much effort is not necessarily "obviously out": > > it depends on how important you judge the results of those effort > > versus the results you could have by spending the effort elsewhere. > > I very muich doubt that the corporate users who are currently worried > about the fast pace of change are interested in any particular > feature, and I don't think that any backported feature is going to > make that previous version more popular amongst managers with decision > power. The question in front of them is, "should we use Python or > not", not "should we use Python 2.1". To answer that question, they > want to perceive "Python" as stable, not "Python 2.1". True, and yet such decision makers DO want to perceive that the specific software they use IS actively supported. It's a reasonable desire indeed, as I've tried to explain quite a few times. If they perceive that choosing "Python in general" means they have to choose between an "old, not actively supported any more" version of the language, and one that breaks previously working code every six months, then that will weigh on their mind as a big minus for Python. If they perceived they could choose a "stable but actively supported" version (the existence of an experimental one too would not worry them, I believe -- many popular languages sprout experimental ones based on them too) then that worry would be out of the way, and I'd have a better chance to get them to LOOK at the huge productivity improvements Python has in wait for them... Alex From guido@python.org Mon Apr 8 19:42:04 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 14:42:04 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 11:30:14 PDT." <20020408113014.A24764@glacier.arctrix.com> References: <20020408180043.GA2726@crystal.mems-exchange.org> <20020408113014.A24764@glacier.arctrix.com> Message-ID: <200204081842.g38Ig4k13782@pcp742651pcs.reston01.va.comcast.net> > I agree. Linux is not a good role model. I think FreeBSD has done a > much better job with their -STABLE and -CURRENT branch naming. I don't know how they do it. Can you explain? --Guido van Rossum (home page: http://www.python.org/~guido/) From aleax@aleax.it Mon Apr 8 19:42:04 2002 From: aleax@aleax.it (Alex Martelli) Date: Mon, 8 Apr 2002 20:42:04 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <20020408113014.A24764@glacier.arctrix.com> References: <20020408180043.GA2726@crystal.mems-exchange.org> <20020408113014.A24764@glacier.arctrix.com> Message-ID: <02040820420401.14931@arthur> On Monday 08 April 2002 20:30, Neil Schemenauer wrote: > Andrew Kuchling wrote: > > On Mon, Apr 08, 2002 at 07:35:18PM +0200, Alex Martelli wrote: > > >At least, it worked wonderfully for Linux. Which is where I started > > >this whole thread of discussion... > > > > It's not clear that it has [...] > > I agree. Linux is not a good role model. I think FreeBSD has done a > much better job with their -STABLE and -CURRENT branch naming. Personally, I use OpenBSD where stability and security really matter to me (firewalls, servers) -- and Linux on interactive workstations, because in that case "support for the latest doodad" and "graphical ease in installation and management" matter more to me. But while the technical issues can be argued, I don't think there's any doubt that Linux, starting from nowhere, HAS gained a huge "market share" of hearts and minds among free Unix variants (indeed, among Unix variants of all ilks). The stable/experimental "message" (murkily carried by parity of minor release) seems to have worked better than the stable/current one (with its implication that the stable one is "not current"... which feels BAD to prospective adopters...:-). Alex From aleax@aleax.it Mon Apr 8 19:45:26 2002 From: aleax@aleax.it (Alex Martelli) Date: Mon, 8 Apr 2002 20:45:26 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: Message-ID: <02040820452602.14931@arthur> On Monday 08 April 2002 20:13, Paul Svensson wrote: ... > >whatever. If we can come up with something better than parity > >of minor release number, I'll personally cheer... I just can't think > >of anything better right now. Comparison of microrelease with > >some arbitrary threshold changing by minor.major is worse, though -- > >not a clear signal at all. > > Why not take advantage of the "well known fact" that any software > release ending in ".0" is not to be trusted ? > > Make X.Y.0.N the experimental branch, > settling down to X.Y.1 as the final stable release, > following up with X.Y.Z as bugfix-only releases, > and the next round of experimentals continuing from X.(Y+1).0.0. > > This is almost as simple as the "just call it stable when it is" > proposal, while still being even more obvious than the odd/even approach. I agree it's somewhat more obvious than odd/even, so, although I do not see what you mean about simplicity here, I'd be quite satisfied with it. Alex From paul@svensson.org Mon Apr 8 19:54:37 2002 From: paul@svensson.org (Paul Svensson) Date: Mon, 8 Apr 2002 14:54:37 -0400 (EDT) Subject: [Python-Dev] Stability and change In-Reply-To: <200204081836.g38IaQr13685@pcp742651pcs.reston01.va.comcast.net> Message-ID: On Mon, 8 Apr 2002, Guido van Rossum wrote: >> Make X.Y.0.N the experimental branch, >> settling down to X.Y.1 as the final stable release, >> following up with X.Y.Z as bugfix-only releases, >> and the next round of experimentals continuing from X.(Y+1).0.0. > >This is attractive, if we can make sure that everything settles >between 2.x.0 and 2.x.1. That leaves us only alpha and beta releases >of 2.x.1 to experiment. Could be a long string of betas... The whole idea is that N can grow as large as you like it, before you declare the release stable, and rename it. /Paul From fredrik@pythonware.com Mon Apr 8 20:02:19 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Mon, 8 Apr 2002 21:02:19 +0200 Subject: [Python-Dev] Re: Stability and change References: <20020408180043.GA2726@crystal.mems-exchange.org> <20020408113014.A24764@glacier.arctrix.com> <200204081842.g38Ig4k13782@pcp742651pcs.reston01.va.comcast.net> Message-ID: <009b01c1df2f$ef098500$ced241d5@hagrid> Guido wrote: > > > I agree. Linux is not a good role model. I think FreeBSD has done a > > much better job with their -STABLE and -CURRENT branch naming. > > I don't know how they do it. Can you explain? develop against the main trunk (CURRENT), and release from a branch (STABLE). more here: http://www.freebsd.org/doc/en_US.ISO8859-1/articles/releng/index.html debian uses UNSTABLE, TESTING, and STABLE. http://www.debian.org/releases/ From aleax@aleax.it Mon Apr 8 20:10:57 2002 From: aleax@aleax.it (Alex Martelli) Date: Mon, 8 Apr 2002 21:10:57 +0200 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: <200204081755.g38HtXt13343@pcp742651pcs.reston01.va.comcast.net> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081755.g38HtXt13343@pcp742651pcs.reston01.va.comcast.net> Message-ID: <02040821105703.14931@arthur> On Monday 08 April 2002 19:55, Guido van Rossum wrote: ... > > On the other hand, software dev't managers don't like the choice > > between an "old, obsolete, probably not of interest any more" piece > > of software, and one that's "so brand-new it's still churning AND > > may break your existing correct code". > > But that's still the choice they get, albeit phrased differently. If that's indeed all the choice they get, then phrasing, aka spin, is only going to help a little, and transiently. If the stable release is to be successfully marketed as "stable but actively maintained" then the part after the "but" needs to have some truth to it. However, I think that a clear message in this regard may half- magically help make itself true -- to wit: > > afford a better chance. The "stable" track would focus on not > > breaking existing correct programs; the "experimental" track would > > focus on enhancements with more freedom (including freedom to > > take some language-design risks). > > I'm not sure the stable track would differ in practice from what we're > already doing with 2.1.3 and 2.2.1. I think the clear separation would help. Consider a book author: what release is he or she going to focus on? He or she clearly wants to target the release that is CURRENT when the book comes out -- but need not target a release that is tagged as EXPERIMENTAL, unless perhaps in an advanced book. That is how the book market works: there's much less demand for a book about a release of software that is not current any more -- or about a release that is experimental. Thus the clear message "this is current, stable, actively maintained", even by itself, is going to attract some more volunteer active maintenance -- not quite a self-fulfilling prophecy, but... it does not need to follow that the experimental release gets less -- if the pie grows, both slices may get larger. Besides, "experimental" has its own allure -- you could call it "leading-edge" internally:-). > I don't know that 1.5.2's stability attracted people. It was probably > more its feature-fullness, plus the fact that around the same time > lots of books etc. started to appear. _I_ was attracted by a perception of 1.5.2's stability -- I'm rather a late-comer, remember. I thought a stable language gave me better chances to convince what was then my management -- and like many others I did not understand Python was so much stabler than (e.g. Perl), not for a deliberate decision, but by happenstance. I've heard others make very similar remarks in various personal conversations -- that the perceived stability was part of the appeal. (Some told me exactly the reverse, or rather explained why they chose, say, Ruby -- perceived as more active and therefore easier to make a mark in; I _suspect_ that if an experimental track had existed, Python would have looked more attractive to them without thereby losing attractiveness to the first crowd). Alex From andymac@bullseye.apana.org.au Mon Apr 8 14:04:36 2002 From: andymac@bullseye.apana.org.au (Andrew MacIntyre) Date: Tue, 9 Apr 2002 00:04:36 +1100 (edt) Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido, As I see it:- - you're not going to please everybody. - you're not even going to please most people. - you'll be lucky to please a good proportion (say a quarter to a third) of your active user (as in developers writing Python apps) base. All I think you'll be able to do:- - try and make your development plans and release functionality goals more explicit and widely known; - use your limited development resources in much the same way as you currently do; - make a decision as to whether fixing warts at the cost of backward compatibility is better than preserving warts in perpetuity. [long-winded rambling attempt to support above statements...] It all comes down to perception. Release version numbers are both a blessing and a curse, which is why Microsoft has tried desperately to find some other code to identify releases without implying functional performance or API compatibility. There is this perception that a "minor" release number has a specific meaning, and a "patch" or "fix" release number has an even more specific meaning. We even have a PEP spelling it out. In the scheme of things, you may have had a somewhat different response had the following version numbers been used instead: 2.0 => 2.0 2.0.1 => 2.1 2.1 => 2.5 2.1.1 => 2.6 2.1.2 => 2.7 2.1.3 => 2.8 2.2 => 3.0 2.2.1 => 3.1 2.3 => 4.0 (or perhaps 3.5) I'm not saying the response would be more favourable, just different, as different people would read different things from this different set of tea-leaves. Part of the perception problem that I think Paul Rubin and others are exhibiting symptoms of is the implied development road-map associated with version numbering. A corporate software development effort has to take a longish time to consolidate its development plans, and match them up with code developed and bugs/misfeatures/limitations resolved. Hence a 2-3 year major release cycle, with the odd bug-fix release to pacify those on maintenance contracts. People steeped in this sort of corporate ecology find this stability comforting, and the apparently well defined development roadmap reassuring. Because Python isn't the centre of a commercial development effort, its development isn't as well defined as some might expect. Indeed, as with most open source projects, its developers scratch their own itches because they aren't being paid to scratch someone elses. Python is also subject to much tighter resource constraints than I would expect of any commercial effort to do the same task. Consequently, the development plans are more fluid and the delivery schedule more variable. There is also the sense that completed work should be released for use, rather than being hoarded until it fits in with the "grand plan" of the major release cycle. This is in keeping with the "release early, release often" mantra espoused by ESR as an ethic of open source development - the Bazaar. But its antithetical to a corporate ecology - the Cathedral. On the resourcing topic, despite your offers of the keys to the "stable" branches in CVS to anyone who might wish take them on, the only result has been individuals stepping up and doing one "fix" release and then retiring burnt out. This points to the extreme cost of managing "bug fixes" only releases in the midst of active development. Something those who demand such releases (especially those without a maintenance contract to fund the work) don't and probably won't acknowledge. While Alex's "stable" and "development" branch thinking is attractive, I don't believe that the current or likely future resources can sustain it. I also believe that it would be counterproductive to even try it out, in that the resources expended in the effort are IMHO must unlikely to produce any usable return. There is also the matter of backward compatibility. My experience has been that beyond a certain point, the burden of maintaining backward compatibility raises the cost of further enhancement to unacceptable levels. As I prefer to keep making progress, rather than be forever looking back into the past, I expect that backward compatibility should be balanced against the need for progress, and is therefore not sacrosanct though it should be retained where possible/practical. I've already wasted too much of my own resources on this (not to mention yours or those of anyone who has read all the above), so I'll just close by saying that I admire your juggling act, and have faith in what you're doing with Python, even though there are small (in the scheme of things) quibbles I have with some of your decisions. Regards, Andrew. -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au | Snail: PO Box 370 andymac@pcug.org.au | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From guido@python.org Mon Apr 8 20:26:57 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 15:26:57 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 20:37:29 +0200." <02040820372900.14931@arthur> References: <0204081020040G.05787@arthur> <200204081818.g38IIMe13541@pcp742651pcs.reston01.va.comcast.net> <02040820372900.14931@arthur> Message-ID: <200204081926.g38JQvt14026@pcp742651pcs.reston01.va.comcast.net> [Alex, getting better at brevity] > Judging by languages such as C or C++, stability seems paramount; > yet Java churned a lot and still managed to spread a lot too (not > without a lot of help from high-$$$ marketing efforts, though). Another example of the Logajan paradox. > True, and yet such decision makers DO want to perceive that the > specific software they use IS actively supported. It's a reasonable > desire indeed, as I've tried to explain quite a few times. Yet it is close to Logajan's position. > If they perceive that choosing "Python in general" means they have > to choose between an "old, not actively supported any more" version > of the language, and one that breaks previously working code every > six months, then that will weigh on their mind as a big minus for > Python. So it's purely a matter of spin. Because new Python releases every 6 months do *not* mean that code breaks every 6 months. Yet some people continue to believe this. > If they perceived they could choose a "stable but actively > supported" version (the existence of an experimental one too would > not worry them, I believe -- many popular languages sprout > experimental ones based on them too) then that worry would be out of > the way, and I'd have a better chance to get them to LOOK at the > huge productivity improvements Python has in wait for them... Maybe all we need to do is make the micro releases a bit more visible... --Guido van Rossum (home page: http://www.python.org/~guido/) From aleax@aleax.it Mon Apr 8 20:24:54 2002 From: aleax@aleax.it (Alex Martelli) Date: Mon, 8 Apr 2002 21:24:54 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: Message-ID: <02040821245404.14931@arthur> On Monday 08 April 2002 19:52, Christopher Petrilli wrote: ... > In the end, I'd like to see something more concrete than speculation if > we're going to change behavior. So would I -- but I have no budget for a market survey, so I can only offer my modest personal experience. I feel reasonably sure, about quite a few development organizations I've been in touch with, that I would have a _much_ better chance to convince them to try Python in earnest if I could send the "stability and change" message. A couple of those organizations I know rather intimately (for one of them, I've been Senior Software Consultant for 12+ years) and thus it's not just an "I feel reasonably sure"... I *would* bet heavily that the change would be very important in that case. As to why you think such potential adopters, clearly so resistant to change, would churn MORE than people who don't mind change, that goes against common sense, so I think the burden of proof should be on you in this case. Why would a change-resistant user change back more easily than a change-favourable user? That's contradictory. > I'm not opposed to a new model, I just > want to understand what it gains in more concrete terms. Speculation as > to what the user wants gets you Microsoft Bob. On the contrary: Microsoft has money to burn (particularly had it back when Bob was being hatched), so the failure of that project, to me, shows the modest value of market surveys (limited to the case of products meant to be highly innovative -- it says nothing either pro or con the value of such surveys when applied to very different issues such as the one that involves us now, of course). Alex From fdrake@acm.org Mon Apr 8 20:24:39 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 8 Apr 2002 15:24:39 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15537.55112.191870.654238@anthem.wooz.org> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> Message-ID: <15537.61047.435656.443349@grendel.zope.com> Barry A. Warsaw writes: > I'd dread having to explain to people "this is compatible with Python > Python 2.4.2 but not anything earlier than 2.4.1 unless you're on the > 2.3.x branch and then 2.3.7 is okay, but not 2.3.6 or earlier." Yes; that's a PR pain. I think someone already suggested that when a feature-release series is determined to be "stable" (however that gets defined), a new stable series is started. I.e., if we determine X.Y.Z is stable, it gets released as X.(Y+1), and feature development moves to X.(Y+2). That's pretty much Linux-style as far as I can tell. I don't like the magic interpretation of even/odd, but I can live with that. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From gmcm@hypernet.com Mon Apr 8 20:39:47 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Mon, 8 Apr 2002 15:39:47 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081809.g38I9EY13505@pcp742651pcs.reston01.va.comcast.net> References: Your message of "Mon, 08 Apr 2002 12:31:06 EDT." Message-ID: <3CB1B9C3.8307.5D896290@localhost> On 8 Apr 2002 at 14:09, Guido van Rossum wrote: > > Are the conservative forces truly the majority, or > > are they simply the most vocal? > > I'd be very surprised if they weren't in the > majority. It's pretty much what the word > conservative *means*, right? It means being cautious. It's a label that reactionaries love to hijack, so they can paint the (merely) cautious as "liberals" and anyone with a new idea as "lunatic fringe". Conservatives will stay quietly on an older release. Reactionaries will post to c.l.py. -- Gordon http://www.mcmillan-inc.com/ From gmcm@hypernet.com Mon Apr 8 20:39:46 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Mon, 8 Apr 2002 15:39:46 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: Message-ID: <3CB1B9C2.27692.5D896236@localhost> Using Alex Martelli's post as an excuse to mouth off: > ... people who won't use Python > because they don't perceive it as stable enough for > their needs ... Personally, I'd take the stability argument as cover for "I've already got a favorite language", with a good chance that language is VBScript, because Perl/TCL/Java-heads would pick on something else. A number of the loudmouths on c.l.py have outlooks noticebly tinged by paranoia. The others remind me of those who perch for hours around the water-cooler, talking about how they're "working their butts off". Other than a gap right after 1.5.2, Python has done very well with backwards compatibility. There is perhaps too much noise about new features: you guys get (understandably) excited; the water-cooler guys get delicious morsels to feed their delusional tendencies. I care not one whit what numbering scheme is used, as long as it approximates being well-ordered . I do think PLabs should take the good end of the stick, and be the ones to apply the "stable" label. To maintain credibility in this perception-based framework, that probably means being a wee bit behind. (I rather like Alex's idea of a stable track and an experimental track where new features can settle down for awhile.) -- Gordon http://www.mcmillan-inc.com/ From aleax@aleax.it Mon Apr 8 20:49:28 2002 From: aleax@aleax.it (Alex Martelli) Date: Mon, 8 Apr 2002 21:49:28 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081926.g38JQvt14026@pcp742651pcs.reston01.va.comcast.net> References: <02040820372900.14931@arthur> <200204081926.g38JQvt14026@pcp742651pcs.reston01.va.comcast.net> Message-ID: <02040821492805.14931@arthur> On Monday 08 April 2002 21:26, Guido van Rossum wrote: ... > > True, and yet such decision makers DO want to perceive that the > > specific software they use IS actively supported. It's a reasonable > > desire indeed, as I've tried to explain quite a few times. > > Yet it is close to Logajan's position. I dispute that, even though he himself appears to be judging from one post he made to c.l.p -- which I answered trying to dispell his false hopes in the matter. He wants to ensure that software that runs on (e.g.) 2.(N+1) must also have run on 2.N : this is unreasonable, and I've never heard anybody else ask for that. He also seems to want this to hold across major release number changes, 1.* to 2.* -- and again that does not seem to be a widespread priority. > > If they perceive that choosing "Python in general" means they have > > to choose between an "old, not actively supported any more" version > > of the language, and one that breaks previously working code every > > six months, then that will weigh on their mind as a big minus for > > Python. > > So it's purely a matter of spin. Because new Python releases every 6 > months do *not* mean that code breaks every 6 months. Yet some people > continue to believe this. I may be wrong, but I think that the fact that doctests DO break with every release has something to do with this perception. doctest is SO nice, easy and natural to use (within a single release) that it's hard to accept it as "unusable between minor releases", even though I have to reluctantly accept that decision. > > If they perceived they could choose a "stable but actively > > supported" version (the existence of an experimental one too would > > not worry them, I believe -- many popular languages sprout > > experimental ones based on them too) then that worry would be out of > > the way, and I'd have a better chance to get them to LOOK at the > > huge productivity improvements Python has in wait for them... > > Maybe all we need to do is make the micro releases a bit more > visible... I may be wrong, but I think the concept of dual, stable and experimental, tracks, already familiar and accepted from other pieces of software, would facilitate prospective user's perception of "stable but actively supported software". Alex From guido@python.org Mon Apr 8 21:00:24 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 16:00:24 -0400 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: Your message of "Mon, 08 Apr 2002 21:10:57 +0200." <02040821105703.14931@arthur> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081755.g38HtXt13343@pcp742651pcs.reston01.va.comcast.net> <02040821105703.14931@arthur> Message-ID: <200204082000.g38K0O914299@pcp742651pcs.reston01.va.comcast.net> [Guido] > > I'm not sure the stable track would differ in practice from what we're > > already doing with 2.1.3 and 2.2.1. [Alex] > I think the clear separation would help. Consider a book author: what > release is he or she going to focus on? He or she clearly wants to > target the release that is CURRENT when the book comes out And there's the crux -- that one is still EXPERIMENTAL while the book is being written. > -- but need not target a release that is tagged as EXPERIMENTAL, > unless perhaps in an advanced book. That is how the book market > works: there's much less demand for a book about a release of > software that is not current any more -- or about a release that is > experimental. > Thus the clear message "this is current, stable, actively maintained", > even by itself, is going to attract some more volunteer active > maintenance -- not quite a self-fulfilling prophecy, but... it does not > need to follow that the experimental release gets less -- if the pie > grows, both slices may get larger. Besides, "experimental" has its > own allure -- you could call it "leading-edge" internally:-). So which distinction should we make? In a previous msg you disliked STABLE vs. CURRENT. Would you prefer STABLE vs. EXPERIMENTAL or CURRENT vs. EXPERIMENTAL? I note that for things like FreeBSD or Debian (or Linux, for that matter) the issues are fundamentally different than for Python. It's a matter of scale, really: an OS consists of many, many packages with independant developers, and packages (more likely, specific versions of packages) get "promoted" to the "stable" stream through some kind of "vetting" process. While this might resemble how some of Python's largest packages (email, XML) are being developed, for the majority of code in Python, both library and core, this is not how it works at all! Experimental stuff occasionally lives in a branch (when it's really big) but more commonly simply begins life as a patch on SF, which is reviewed a few times and then committed; if there's a problem, we try to fix it ASAP. Python is still small enough that a committed bug (something that breaks the unittests) is relatively rare, and cause of great turmoil! We see two kinds of changes to Python: small things, typically bugfixes (often a few lines in one module) that are "obviously" correct (except when they aren't :-); and large things, typically new features, and occasionally refactorings, which touch many files. We're being very careful with the latter not to commit them unless the unittests succeed (adding unittests for new features as necessary). Should we do more? I know the Zope developers (theoretically :-) use branches for all new code (whether feature or refactoring, even for the larger bugfixes), and this seems to be similar to what the FreeBSD STABLE/CURRENT distinction is getting at. But IMO it doesn't work all that well for Zope (merging branches is a pain and we often see bugs introduced by merges) and in general I'm wary of branches that require frequent merging. Maybe there's no way out: as long as the development of the language goes at the pace it goes, there's not much we can do to prevent language users to see frequent releases and frequent (small) changes. If we reduce stable releases to once a year, that's still too frequent for some (Rubin has requested 2-3 years between releases) while others will be so eager to use new features that they'll risk using the CVS head -- and then end up hating it for its instability (because all they want is the "big" new features, not the day-to-day churn). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 8 21:05:05 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 16:05:05 -0400 Subject: [Python-Dev] Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 14:54:37 EDT." References: Message-ID: <200204082005.g38K55e14380@pcp742651pcs.reston01.va.comcast.net> > >> Make X.Y.0.N the experimental branch, > >> settling down to X.Y.1 as the final stable release, > >> following up with X.Y.Z as bugfix-only releases, > >> and the next round of experimentals continuing from X.(Y+1).0.0. > > > >This is attractive, if we can make sure that everything settles > >between 2.x.0 and 2.x.1. That leaves us only alpha and beta releases > >of 2.x.1 to experiment. Could be a long string of betas... > > The whole idea is that N can grow as large as you like it, > before you declare the release stable, and rename it. Unfortunately we have a formally defined encoding for versions that this would break. :-( See patchlevel.h and sys.hexversion. --Guido van Rossum (home page: http://www.python.org/~guido/) From geoff-pdev@gerrietts.net Mon Apr 8 20:58:47 2002 From: geoff-pdev@gerrietts.net (Geoff Gerrietts) Date: Mon, 8 Apr 2002 12:58:47 -0700 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081818.g38IIMe13541@pcp742651pcs.reston01.va.comcast.net> References: <02040710094202.05787@arthur> <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <0204081020040G.05787@arthur> <200204081818.g38IIMe13541@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020408195847.GA10251@isis.gerrietts.net> Quoting Guido van Rossum (guido@python.org): > I very muich doubt that the corporate users who are currently worried > about the fast pace of change are interested in any particular > feature, and I don't think that any backported feature is going to > make that previous version more popular amongst managers with decision > power. The question in front of them is, "should we use Python or > not", not "should we use Python 2.1". To answer that question, they > want to perceive "Python" as stable, not "Python 2.1". I think, as this discussion slowly seems to be winding down and my list subscription goes through, that I can finally contribute something. One of the points I've seen brought across is that "stable python" needs to be more widely acclaimed as such. The example seems to be the 2.1.3 release, or the 2.1.* series in general. I agree with this, wholeheartedly. The remains of the discussion seems to circle around how to number or manage the changes so that it's easier to know how things are developing. To this, I think that picking a scheme is a lot less important than documenting what that scheme means to the people who are using it. Maybe we can cash in on some of the ideas put forward by other people, maybe we can't, maybe we shouldn't. What's important is not how we do it, but that we're explicit about what we do. The remains of this note are going to be long-winded, so if you're prone to start skimming, now would be a good time . BACKGROUND (where I'm coming from) A little background -- I'm part of a company (at my other email address) that views Python as part of their competitive advantage. We engage in weekly rolls into production, with product cycles that last between two and four weeks. Three of us write python regularly; four of us write DTML and HTML. Management duties, release scheduling, release management, deployment, and site maintenance responsibilities are scattered liberally among this crew of seven. We do a lot of business at this site, which makes those scattered responsibilities fairly time-consuming for some of us. Despite this, we manage to maintain the aggressive schedule, which has in turn enabled us to bring in big partnerships and close substantial deals with a very small core of engineers holding it all together. In many ways, we are the sort of story that serves as an advertisement for Python (though you should stop well short of holding up our code as examples, if you know what I mean). On the other hand, a full overhaul of the site to advance to a new version of Python is a substantial project. Between the dependencies on third-party products, the extensive customizations we've had to make, the QA overhead that goes into a full site regression, and the management overhead that goes into tracking down all the bits and pieces -- that's about a month-long project, though we might be able to fit it into 3 weeks. This puts us into the "conservative" camp. For some time now, we have regarded it easier to steer around, work around, or hack around bugs in 1.5.2 than to upgrade. We've backported a fix or two. It's still not enough. Recently, we've agreed to upgrade, and managed to secure executive permission to take a 3 week cycle (scattered over the next couple months) to do the updating work. My situation may be atypical, but I think that we're the sort of operation that the python community doesn't really want to alienate. THOUGHTS ON STABILITY When I use python for myself, I prefer to use 2.2, because I prefer to stay current. When I develop python for a wider audience, I plan for 1.5.2, because that's what I'm pretty sure I can count on people to have available. I can speculate why this is so, but it would be speculation. When I looked at upgrading to the 2.* series of python interpreters, I wasn't sure where to target. 2.0 is for all intents and purposes a dead end. 2.1 is seeing some active bugfix development, but there's no visible commitment from the community to continue that trend. In fact, when the topic gets broached, it more often than not gets shot down with "not likely, who would want to waste their time on doing that?" or some variation therof. 2.2 appears stable, but there's a bugfix release in the works that hasn't (as far as I know) been made official yet. In other words, none of the 2.* series appeared like particularly attractive targets to shoot at, because all of them appeared to be unsuitable for the needs of our organization. I decided to shoot for 2.2, because it's hard to tell when we'll be able to do the upgrade again, and I wanted to solve as many problems as could be solved, now. It appears that may have been a bad choice, given progress with 2.1.3. People talk about a 6 month gap between releases, but I'm not sure anyone recognizes that timeframe as "average" yet. You can have an average with only two measurements, but it really takes three to make a series, and the gap between 1.5.2 and 2.0 was considerably longer than 6 months. The point here is that when you adopt any given minor version of python, you have no idea how long you have before you're obsolescent. Six months is reasonable, if that is in fact what we continue to hit. But being predictable is more important to "the conservative crowd" than any particular interval could be. In conclusion, I think that what "the conservatives" are looking for is really quite easy to provide. I summarize them as: - Be predictable. If we're putting off pain until 3.0, make a guess when we're going to need to deal with that pain, so we can start preparing. If 3.0 is 9 months off, that's a lot scarier and more concerning than if it's 2 years off. - Be explicit about the process. If the process is to stick a fork in the ham and call it "stable" just before moving on to the next minor release, then be explicit about that process. - Be explicit about obsolescence. If things in 2.3 or 3.0 or X.Y are shaping up to be so different that bugfixes won't be back-portable, then identify that fact early -- and make an effort to provide a reasonable target for people to upgrade to, as soon as possible. The perceived problem with change being too rapid may be just that, a perceived problem. With adequate and careful communication, the problem may disappear entirely. It may not, but we can always dream. :) Anyway, that's about all I can contribute, after thinking long and seriously about what I feel like our company needs, and trying to ignore the specific details of implementation. I hope it's helpful. Thanks, --G. -- Geoff Gerrietts http://www.gerrietts.net/ "Politics, as a practice, whatever its professions, has always been the systematic organization of hatreds." --Henry Adams From guido@python.org Mon Apr 8 21:13:13 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 16:13:13 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 21:24:54 +0200." <02040821245404.14931@arthur> References: <02040821245404.14931@arthur> Message-ID: <200204082013.g38KDDc14469@pcp742651pcs.reston01.va.comcast.net> [Alex] > I feel reasonably sure, about quite a few development organizations > I've been in touch with, that I would have a _much_ better chance to > convince them to try Python in earnest if I could send the > "stability and change" message. The more I think about it, the more I believe that there's no reason we couldn't send them that message *now*, if it weren't for a few corncobs on c.l.py. --Guido van Rossum (home page: http://www.python.org/~guido/) From aleax@aleax.it Mon Apr 8 21:23:20 2002 From: aleax@aleax.it (Alex Martelli) Date: Mon, 8 Apr 2002 22:23:20 +0200 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: <200204082000.g38K0O914299@pcp742651pcs.reston01.va.comcast.net> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <02040821105703.14931@arthur> <200204082000.g38K0O914299@pcp742651pcs.reston01.va.comcast.net> Message-ID: <02040822232006.14931@arthur> On Monday 08 April 2002 22:00, Guido van Rossum wrote: > [Guido] > > > > I'm not sure the stable track would differ in practice from what > > > we're already doing with 2.1.3 and 2.2.1. > > [Alex] > > > I think the clear separation would help. Consider a book author: what > > release is he or she going to focus on? He or she clearly wants to > > target the release that is CURRENT when the book comes out > > And there's the crux -- that one is still EXPERIMENTAL while the book > is being written. That depends on how often the experimental branch is fed back into a new stable one. If every 6 months, well, sure. But I think it would not need to be anywhere as frequent. > > Thus the clear message "this is current, stable, actively maintained", > > even by itself, is going to attract some more volunteer active > > maintenance -- not quite a self-fulfilling prophecy, but... it does not > > need to follow that the experimental release gets less -- if the pie > > grows, both slices may get larger. Besides, "experimental" has its > > own allure -- you could call it "leading-edge" internally:-). > > So which distinction should we make? In a previous msg you disliked > STABLE vs. CURRENT. Would you prefer STABLE vs. EXPERIMENTAL or > CURRENT vs. EXPERIMENTAL? I think both sound very good in the abstract (while stable vs current does not). It seems to me that stable vs experimental is slightly preferable because it appears to draw a neater distinction, and some connection of 'current' to other-than-stable might otherwise attach from (e.g.) the BSD terminology. But there might be some other consideration that doesn't come to my mind right now that makes current/experimental preferable. > I note that for things like FreeBSD or Debian (or Linux, for that > matter) the issues are fundamentally different than for Python. It's Yes -- different scale, more fragmentation, etc. > Maybe there's no way out: as long as the development of the language > goes at the pace it goes, there's not much we can do to prevent > language users to see frequent releases and frequent (small) changes. > If we reduce stable releases to once a year, that's still too frequent > for some (Rubin has requested 2-3 years between releases) while others > will be so eager to use new features that they'll risk using the > CVS head -- and then end up hating it for its instability (because all > they want is the "big" new features, not the day-to-day churn). There may not be ONE way out -- but maybe TWO tracks would do, instead. On the stable track, releases that could break previously correct code would be highly infrequent; on the experimental ones, releases would be frequent -- ideally frequent enough that people wouldn't go to CVS unless they're interested in contributing to the internals, but could feel they're "on top of the leading-edge RELEASE" (aka the experimental-track release). No, that would NOT satisfy those who insist on 2.(N+1) not running any code that 2.N didn't -- or worse, any code that 1.5.2 didn't. I don't think there is ANY way to satisfy this request, except burying "Python" and giving a new name to the language (Monty?-). We can't satisfy EVERYbody, I think. But I hope we won't take this as an excuse to do nothing at all:-). Alex From jafo@tummy.com Mon Apr 8 21:25:23 2002 From: jafo@tummy.com (Sean Reifschneider) Date: Mon, 8 Apr 2002 14:25:23 -0600 Subject: [Python-Dev] Pondering some changes to python.c... In-Reply-To: <3CB155D1.422AD8C3@lemburg.com>; from mal@lemburg.com on Mon, Apr 08, 2002 at 10:33:21AM +0200 References: <20020407042038.O16962@tummy.com> <3CB155D1.422AD8C3@lemburg.com> Message-ID: <20020408142523.T16962@tummy.com> On Mon, Apr 08, 2002 at 10:33:21AM +0200, M.-A. Lemburg wrote: >-1 on making it the default -- there are programs relying on this. Interesting point... One of the main reasons that I wanted this was that some programs rely on it *NOT* being there. Like "killall", etc... >+1 on providing a sys module interface for this so that programs >can choose the name on their own. Yeah, I was thinking about that one. It's kind of complicated, unfortunately. To do it right, you need to muck around with the original argv and envp arguments. So, it will cause side-effects to any code calling the main Python entry point if we do it in there... Besides, we really end the envp anyway (which Py_Main doesn't take). So, I'm thinking there would need to be a new entry point like "Py_ArgsSetup(int argc, char ***argv, char ***envp)", which would need to be called to set this up. You'd call "Py_ArgsSetup(argc, &argv, &envp)", and it would re-allocate argv and envp, and use the original data space for storing the process string. Thoughts? Sean -- I have never been able to conceive how any rational being could propose happiness to himself from the exercise of power over others. -- Jefferson Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From skip@pobox.com Mon Apr 8 21:29:29 2002 From: skip@pobox.com (Skip Montanaro) Date: Mon, 8 Apr 2002 15:29:29 -0500 Subject: [Python-Dev] A dizzying stream of checkins Message-ID: <15537.64937.745224.846065@12-248-41-177.client.attbi.com> This is just an aside, only peripherally related to the current "stability" thread, but my head starts to spin when I see a rapid flurry of checkins. (I count 43 in my current inbox since about 6PM EDT Sunday). That doesn't count any I've already deleted. It appears that there are some things going in on the main branch, Guido's checking in bool() and friends for 2.2.1 and Anthony's putting the 2.1.3 release machine in motion. I guess I could sort it out better if I paid closer attention to the presence or absence of a Tag: attribute in the checkin mail. Has anyone given any thought to how this flood of mail might be managed more cleanly? I can certainly mess around locally with procmail, but I was wondering if it was worth trying to do something more centrally. Skip From guido@python.org Mon Apr 8 21:43:55 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 16:43:55 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 12:58:47 PDT." <20020408195847.GA10251@isis.gerrietts.net> References: <02040710094202.05787@arthur> <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> <0204081020040G.05787@arthur> <200204081818.g38IIMe13541@pcp742651pcs.reston01.va.comcast.net> <20020408195847.GA10251@isis.gerrietts.net> Message-ID: <200204082043.g38Khuk14801@pcp742651pcs.reston01.va.comcast.net> [Geoff Gerrietts] > When I looked at upgrading to the 2.* series of python interpreters, > I wasn't sure where to target. 2.0 is for all intents and purposes a > dead end. 2.1 is seeing some active bugfix development, but there's > no visible commitment from the community to continue that trend. In > fact, when the topic gets broached, it more often than not gets shot > down with "not likely, who would want to waste their time on doing > that?" or some variation therof. 2.2 appears stable, but there's a > bugfix release in the works that hasn't (as far as I know) been made > official yet. In other words, none of the 2.* series appeared like > particularly attractive targets to shoot at, because all of them > appeared to be unsuitable for the needs of our organization. I > decided to shoot for 2.2, because it's hard to tell when we'll be > able to do the upgrade again, and I wanted to solve as many problems > as could be solved, now. It appears that may have been a bad choice, > given progress with 2.1.3. If you're coming from 1.5.2, picking 2.2, the most recent "final" release, is the right choice. Most breakage of 1.5.2 code was done in 2.0, when we didn't have warnings in place, and added Unicode. 2.1.3 is good for people who ar already on 2.1 and who want to see some bugs fixed. We fixed TONS of bugs in 2.1.1 and 2.1.2; 2.1.3 will be released tonight with only a very small number of additional fixes, but including one that happens to be important for some Zope users (Zope can do some weird shit that you're not likely to run into yourself :-). 2.2.1 will *also* be released tonight. It fixes most known bugs in 2.2, adding almost no new features (depending on how the size of your yardstick). The idea is that anybody who would want to run 2.2 would be even happier with 2.2.1, and that people who were afraid that 2.2 would crash an burn can now try out 2.2 without that fear. > People talk about a 6 month gap between releases, but I'm not sure > anyone recognizes that timeframe as "average" yet. You can have an > average with only two measurements, but it really takes three to > make a series, and the gap between 1.5.2 and 2.0 was considerably > longer than 6 months. The point here is that when you adopt any > given minor version of python, you have no idea how long you have > before you're obsolescent. Six months is reasonable, if that is in > fact what we continue to hit. But being predictable is more > important to "the conservative crowd" than any particular interval > could be. Before 1.5.2, we *also* hit 6 months with some regularity. The only irregularity was really the 18 month gap between 1.5.2 and 2.0. We're seeing the gap slowly increase, to maybe 8 months for 2.3, but I expect that 6-8 months is probably what we'll continue to see. > In conclusion, I think that what "the conservatives" are looking for > is really quite easy to provide. I summarize them as: > > - Be predictable. If we're putting off pain until 3.0, make a guess > when we're going to need to deal with that pain, so we can start > preparing. If 3.0 is 9 months off, that's a lot scarier and more > concerning than if it's 2 years off. 3.0 will always be 2 years off. :-) At least for now, it really is. > - Be explicit about the process. If the process is to stick a fork in > the ham and call it "stable" just before moving on to the next minor > release, then be explicit about that process. We are already pretty explicit; there are several PEPs about the process. > - Be explicit about obsolescence. If things in 2.3 or 3.0 or X.Y are > shaping up to be so different that bugfixes won't be back-portable, > then identify that fact early -- and make an effort to provide a > reasonable target for people to upgrade to, as soon as possible. It depends on the bugfix. Some modules and subsystems are being refactored in order to provide new features, and then bugfixes won't port back. Other modules and subsystems are very stable (some could be called stagnant :-) and will have no problem getting bugfixes backported. > The perceived problem with change being too rapid may be just that, a > perceived problem. With adequate and careful communication, the > problem may disappear entirely. It may not, but we can always dream. > :) Yes, maybe it can be made to disappear. Does that mean we need to throw the reactionaries on c.l.py a bone? Is this really all politics? In that case I'm out of here. > Anyway, that's about all I can contribute, after thinking long and > seriously about what I feel like our company needs, and trying to > ignore the specific details of implementation. I hope it's helpful. Thanks! --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Mon Apr 8 21:42:42 2002 From: skip@pobox.com (Skip Montanaro) Date: Mon, 8 Apr 2002 15:42:42 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15538.194.350128.498326@12-248-41-177.client.attbi.com> Alex> It seems to me that duplicating (in your example) 2.3.8 to 2.4.0 Alex> (and using 2.5.0 as the new baseline for further experimentation) Alex> would be a very clear signal in this sense. This is essentially the Linux kernel release model, yes? Once 2.3.8 morphs simultaneously into 2.4.0 and 2.5.0, the 2.3.x branch dies. New development goes into 2.5.0 and bugfixes go into both branches, as appropriate. I think an important question is what happens to the 2.2.x branch once 2.4.0 is released? Should it die (in the sense of *never* getting another micro release)? I think that would be a fair approach, otherwise you have an ever-increasing support burden, trying to handle more and more releases. Was there ever a huge clamor for 1.5.3? It seems that for many people the heavens opened and Gabriel descended with a 1.5.2 CD. ;-) Skip From guido@python.org Mon Apr 8 21:46:28 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 16:46:28 -0400 Subject: [Python-Dev] A dizzying stream of checkins In-Reply-To: Your message of "Mon, 08 Apr 2002 15:29:29 CDT." <15537.64937.745224.846065@12-248-41-177.client.attbi.com> References: <15537.64937.745224.846065@12-248-41-177.client.attbi.com> Message-ID: <200204082046.g38KkSp14849@pcp742651pcs.reston01.va.comcast.net> > This is just an aside, only peripherally related to the current > "stability" thread, but my head starts to spin when I see a rapid > flurry of checkins. (I count 43 in my current inbox since about 6PM > EDT Sunday). That doesn't count any I've already deleted. It > appears that there are some things going in on the main branch, > Guido's checking in bool() and friends for 2.2.1 and Anthony's > putting the 2.1.3 release machine in motion. I guess I could sort > it out better if I paid closer attention to the presence or absence > of a Tag: attribute in the checkin mail. Has anyone given any > thought to how this flood of mail might be managed more cleanly? I > can certainly mess around locally with procmail, but I was wondering > if it was worth trying to do something more centrally. I think it may be worse because SF held on to a whole bunch of checkin mails and then decided to send them all at once. Also, we're doing *two* releases tonight. Zope Corp uses a superior post-commit action script that seems to be able to send email to different lists depending to the tag; maybe we can add that feature to our commit script? --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Mon Apr 8 21:45:02 2002 From: skip@pobox.com (Skip Montanaro) Date: Mon, 8 Apr 2002 15:45:02 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081734.g38HYwb13124@pcp742651pcs.reston01.va.comcast.net> References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> <15537.43727.369744.356521@grendel.zope.com> <15537.47467.132247.414879@anthem.wooz.org> <20020408171316.GC2216@crystal.mems-exchange.org> <200204081734.g38HYwb13124@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15538.334.106174.798218@12-248-41-177.client.attbi.com> Guido> I think version dependency management is subject to the Logajan Guido> paradox. :-( Can you state that succintly for us? :-) S From guido@python.org Mon Apr 8 21:51:57 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 16:51:57 -0400 Subject: [Python-Dev] Pondering some changes to python.c... In-Reply-To: Your message of "Mon, 08 Apr 2002 14:25:23 MDT." <20020408142523.T16962@tummy.com> References: <20020407042038.O16962@tummy.com> <3CB155D1.422AD8C3@lemburg.com> <20020408142523.T16962@tummy.com> Message-ID: <200204082051.g38Kpv314922@pcp742651pcs.reston01.va.comcast.net> > >-1 on making it the default -- there are programs relying on this. > > Interesting point... One of the main reasons that I wanted this was that > some programs rely on it *NOT* being there. Like "killall", etc... When I run a Python script that has a #! directly, ps doesn't show me the "python" part -- only the script name. This is in Linux (Red Hat 7.2). Maybe it's different for you? I think that it's totally acceptable that when you say "python foo.py" that argv[0] is seen by ps to be "python". > >+1 on providing a sys module interface for this so that programs > >can choose the name on their own. > > Yeah, I was thinking about that one. It's kind of complicated, > unfortunately. To do it right, you need to muck around with the original > argv and envp arguments. So, it will cause side-effects to any code > calling the main Python entry point if we do it in there... Besides, we > really end the envp anyway (which Py_Main doesn't take). > > So, I'm thinking there would need to be a new entry point like > "Py_ArgsSetup(int argc, char ***argv, char ***envp)", which would need to be > called to set this up. You'd call "Py_ArgsSetup(argc, &argv, &envp)", and > it would re-allocate argv and envp, and use the original data space for > storing the process string. I'm not sure I understand. If you have the argc/argv/environ set, why do you need to know &argc and &argv? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 8 21:55:11 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 16:55:11 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 15:45:02 CDT." <15538.334.106174.798218@12-248-41-177.client.attbi.com> References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> <15537.43727.369744.356521@grendel.zope.com> <15537.47467.132247.414879@anthem.wooz.org> <20020408171316.GC2216@crystal.mems-exchange.org> <200204081734.g38HYwb13124@pcp742651pcs.reston01.va.comcast.net> <15538.334.106174.798218@12-248-41-177.client.attbi.com> Message-ID: <200204082055.g38KtCU14969@pcp742651pcs.reston01.va.comcast.net> > Guido> I think version dependency management is subject to the > Guido> Logajan paradox. :-( > > Can you state that succintly for us? :-) Logajan wants all code he wrote for 1.5.2 to work in 2.0, 2.1, and so on. He also complains that a script written for 2.1 doesn't work in 2.0 (and no, that's not reversed!). I conclude that he doesn't want the language to change at all. --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Mon Apr 8 21:51:36 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 8 Apr 2002 16:51:36 -0400 Subject: [Python-Dev] A dizzying stream of checkins In-Reply-To: <200204082046.g38KkSp14849@pcp742651pcs.reston01.va.comcast.net> References: <15537.64937.745224.846065@12-248-41-177.client.attbi.com> <200204082046.g38KkSp14849@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15538.728.638642.489867@grendel.zope.com> Guido van Rossum writes: > Zope Corp uses a superior post-commit action script that seems to be > able to send email to different lists depending to the tag; maybe we > can add that feature to our commit script? Based on the tag? It uses the directory of the checkin as the key, which certainly makes it look like the tag might have been involved in the Zope 3 development. But I'm pretty sure it's tree location only. One nicety would be to get the branch name in the subject line -- that would make it a lot easier to filter, either mechanically of visually. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From aleax@aleax.it Mon Apr 8 21:52:58 2002 From: aleax@aleax.it (Alex Martelli) Date: Mon, 8 Apr 2002 22:52:58 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15538.334.106174.798218@12-248-41-177.client.attbi.com> References: <200204081734.g38HYwb13124@pcp742651pcs.reston01.va.comcast.net> <15538.334.106174.798218@12-248-41-177.client.attbi.com> Message-ID: <02040822525808.14931@arthur> On Monday 08 April 2002 22:45, Skip Montanaro wrote: > Guido> I think version dependency management is subject to the > Logajan Guido> paradox. :-( > > Can you state that succintly for us? :-) "Python keeps being maintained yet it *never* happens that release X (for any X and Y) can run some code that release Y cannot run". It's unachievable (as well as unreasonable) and thus I don't think it should be considered. A desire that I think more reasonable (still unachievable, but one can come closer) is a similar wish stated only for Y > X. In practice we do need break-points where such backwards compatibility cannot be guaranteed ("major releases") -- further, I think that for the wish to be both achievable AND compatible with other important goals it must be nuanced ('some code' becoming 'CORRECT code', for example -- broken code that HAPPENS to run by accident is quite another thing: although for some users the difference can be hard to appreciate, the concept of 'undefined behavior' is, I believe, known widely enough to rely on it a bit, at least where the docs mention it). Alex From skip@pobox.com Mon Apr 8 21:56:28 2002 From: skip@pobox.com (Skip Montanaro) Date: Mon, 8 Apr 2002 15:56:28 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081738.g38HcvF13190@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15538.1020.693595.182661@12-248-41-177.client.attbi.com> Alex> But SOMEthing more remarkable that "up to 2.3.N experimental, Alex> 2.3.(N+1) and on stable" is needed -- a different name, a Alex> different major release, whatever. It seems to me that having functional goals for development releases is the right way to measure functional completeness and stability. If you have this list of goals, that provides a way to directly measure functional completeness. Once a development release is functionally complete, perhaps you can begin to measure the bug fix frequency more-or-less independent of the functional change frequency. Once the branch is functionally complete and bug fix frequency drops "sufficiently" far, you've reached a point where your confidence in splitting into new stable and development branches makes sense. Another motivator might be the presence of impending new development. Suppose MAL has submitted a complete rewrite of Python's Unicode handling , but your currently at 2.3.13 and feeling pretty good about the stability of that code. You fork into 2.4.0 and 2.5.0. MAL's new Unicode implementation can be at the top of the list for 2.5's goals and go into 2.5.x. 2.4.x can then pick up more bug fixes. By the time you reach 2.4.2 or thereabouts, people should be really comfortable with the stability of that code. Skip From jeremy@zope.com Mon Apr 8 21:50:48 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 8 Apr 2002 16:50:48 -0400 Subject: [Python-Dev] A dizzying stream of checkins In-Reply-To: <200204082046.g38KkSp14849@pcp742651pcs.reston01.va.comcast.net> References: <15537.64937.745224.846065@12-248-41-177.client.attbi.com> <200204082046.g38KkSp14849@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15538.680.660947.60291@slothrop.zope.com> >>>>> "GvR" == Guido van Rossum writes: GvR> Zope Corp uses a superior post-commit action script that seems GvR> to be able to send email to different lists depending to the GvR> tag; maybe we can add that feature to our commit script? I think Ken wrote it, and it is very easy to configure. I've never been able to figure out how the checkin -> email process works in much detail, but the traffic_table.py script does what I want without making me understand the details. We could certainly split python-checkins into many lists: all-python-checkins: everything python-checkins: checkins on the trunk python-maint-checkins: checkins on maintenance branchs python-other-checkins: everything else This means that all-python-checkins == (python-checkins + python-maint-checkins + python-other-checkins). Perhaps there should be a separate doc checkin list, but I might be happy leaving them in with the rest of the checkins. Then we could add some specific rules for little corners of the CVS tree. Like nondist/sandbox/ast could be sent to the compiler-sig. Jeremy From tim.one@comcast.net Mon Apr 8 21:55:37 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 08 Apr 2002 16:55:37 -0400 Subject: [Python-Dev] A dizzying stream of checkins In-Reply-To: <200204082046.g38KkSp14849@pcp742651pcs.reston01.va.comcast.net> Message-ID: > ... > Also, we're doing *two* releases tonight. I didn't know that. I'm glad someone mentioned it . > Zope Corp uses a superior post-commit action script that seems to be > able to send email to different lists depending to the tag; maybe we > can add that feature to our commit script? Please let's not try to "fix this" -- they're all Python checkins, and it's trivial to notice the presence or absence of a tag line by eyeball. From fdrake@acm.org Mon Apr 8 21:56:19 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 8 Apr 2002 16:56:19 -0400 Subject: [Python-Dev] 2.2.1 release mechanics In-Reply-To: <2m7knspj43.fsf@starship.python.net> References: <2m7knspj43.fsf@starship.python.net> Message-ID: <15538.1011.400324.537358@grendel.zope.com> Michael Hudson writes: > April 10 ~1200 GMT: release ends > > By now, F, T & J have done their bits, uploaded > files to creosote and sf (or pointed me to where I > can get them), etc. Are we still on target for this? -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From guido@python.org Mon Apr 8 22:00:49 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 17:00:49 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 22:52:58 +0200." <02040822525808.14931@arthur> References: <200204081734.g38HYwb13124@pcp742651pcs.reston01.va.comcast.net> <15538.334.106174.798218@12-248-41-177.client.attbi.com> <02040822525808.14931@arthur> Message-ID: <200204082100.g38L0ol15046@pcp742651pcs.reston01.va.comcast.net> > the concept of 'undefined behavior' is, I believe, known > widely enough to rely on it a bit, at least where the docs mention it). Tell that to the people who write L.append(1, 2, 3) despite that it was never documented. :-( --Guido van Rossum (home page: http://www.python.org/~guido/) From aleax@aleax.it Mon Apr 8 22:01:00 2002 From: aleax@aleax.it (Alex Martelli) Date: Mon, 8 Apr 2002 23:01:00 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204082100.g38L0ol15046@pcp742651pcs.reston01.va.comcast.net> References: <02040822525808.14931@arthur> <200204082100.g38L0ol15046@pcp742651pcs.reston01.va.comcast.net> Message-ID: <0204082301000A.14931@arthur> On Monday 08 April 2002 23:00, Guido van Rossum wrote: > > the concept of 'undefined behavior' is, I believe, known > > widely enough to rely on it a bit, at least where the docs mention it). > > Tell that to the people who write L.append(1, 2, 3) despite that it > was never documented. :-( I do -- if and when I see that (haven't seen it in ages). Technical managers at development shops know about stuff "accidentally working" and are more tolerant of bug-fixes against that than of other backwards-compatibility breaks, in my experience. Alex From skip@pobox.com Mon Apr 8 22:04:43 2002 From: skip@pobox.com (Skip Montanaro) Date: Mon, 8 Apr 2002 16:04:43 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15537.55112.191870.654238@anthem.wooz.org> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> Message-ID: <15538.1515.580133.785503@12-248-41-177.client.attbi.com> Geez, I'm not going to make it to the end of p-dev mail today... BAW> Maybe I misunderstand, but it makes me really nervous that micro BAW> releases would have internal incompatibilities (I'm not talking BAW> about those related to fixed bugs in earlier micros). It depends on what branch you're on. Following up on my previous note about functional completeness, one of your goals might be "to investigate MAL's new Unicode implementation". You might get to 2.5.7 and realize it's not going to work for some reason and then have to rip it back out. Obviously 2.5.8 might then be incompatible with 2.5.7, but since it's the current bleeding edge development branch, you never advertise stability across point releases. Clearly, this sort of toss-it-in-rip-it-out sort of thing wouldn't hold in the Garth branch. BAW> I'd dread having to explain to people "this is compatible with BAW> Python Python 2.4.2 but not anything earlier than 2.4.1 unless BAW> you're on the 2.3.x branch and then 2.3.7 is okay, but not 2.3.6 or BAW> earlier." That's just it. You *never, never, never* require *any* development release to run an external package. If you release a package that requires that sort of coupling, Guido simply calls in a favor from one of his Italian cousins in the Soprano or Corleone families (my apologies for the crude American Mafia-style references, Alex), and they take care of you. :-) Skip From barry@zope.com Mon Apr 8 22:03:39 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Apr 2002 17:03:39 -0400 Subject: [Python-Dev] A dizzying stream of checkins References: <200204082046.g38KkSp14849@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15538.1451.257812.78379@anthem.wooz.org> >>>>> "TP" == Tim Peters writes: TP> Please let's not try to "fix this" -- they're all Python TP> checkins, and it's trivial to notice the presence or absence TP> of a tag line by eyeball. Agreed! Aren't people on enough mailing lists already? -Barry From jafo@tummy.com Mon Apr 8 22:05:23 2002 From: jafo@tummy.com (Sean Reifschneider) Date: Mon, 8 Apr 2002 15:05:23 -0600 Subject: [Python-Dev] Pondering some changes to python.c... In-Reply-To: <200204082051.g38Kpv314922@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Mon, Apr 08, 2002 at 04:51:57PM -0400 References: <20020407042038.O16962@tummy.com> <3CB155D1.422AD8C3@lemburg.com> <20020408142523.T16962@tummy.com> <200204082051.g38Kpv314922@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020408150523.P11182@tummy.com> On Mon, Apr 08, 2002 at 04:51:57PM -0400, Guido van Rossum wrote: >When I run a Python script that has a #! directly, ps doesn't show me >the "python" part -- only the script name. This is in Linux (Red Hat >7.2). Maybe it's different for you? I was told at one point that "#!/usr/bin/env python" was the preferred she-bang incantation. However, calling it directly on my 7.2 box seems to show Python in either case: guin:tmp$ printf '#!/usr/bin/python\nimport time; time.sleep(5)' >testscript guin:tmp$ ./testscript & ps -awlx | grep testscript 000 500 8297 6741 10 0 2256 1192 do_sel S pts/0 0:00 /usr/bin/python ./testscript guin:tmp$ printf '#!/usr/bin/env python\nimport time; time.sleep(5)' >testscript guin:tmp$ ./testscript & ps -awlx | grep testscript 000 500 8300 6741 9 0 2256 1192 do_sel S pts/0 0:00 python ./testscript >I'm not sure I understand. If you have the argc/argv/environ set, why >do you need to know &argc and &argv? On Linux and a few other systems, setting the ps string requires re-writing the block of memory starting at argv[0]. If the data in envp is allocated next to argv, you can also make use of that (so that you can write strings longer than "python testscript" in length). The typical way of doing this involves first making a copy of argv and envp for the process to continue using, because otherwise argv suddenly becomes something entirely different than what it originally was. So, we need to know &argv so that we can hand back the new location. Sean -- The important thing is not to stop questioning. Curiosity has its own reason for existing. -- Albert Einstein Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From barry@zope.com Mon Apr 8 22:06:40 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Apr 2002 17:06:40 -0400 Subject: [Python-Dev] Re: Stability and change References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> <15538.1515.580133.785503@12-248-41-177.client.attbi.com> Message-ID: <15538.1632.252404.964819@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: BAW> I'd dread having to explain to people "this is compatible BAW> with Python Python 2.4.2 but not anything earlier than 2.4.1 BAW> unless you're on the 2.3.x branch and then 2.3.7 is okay, but BAW> not 2.3.6 or earlier." SM> That's just it. You *never, never, never* require *any* SM> development release to run an external package. If you SM> release a package that requires that sort of coupling, Guido SM> simply calls in a favor from one of his Italian cousins in the SM> Soprano or Corleone families (my apologies for the crude SM> American Mafia-style references, Alex), and they take care of SM> you. :-) Okay, so now I tell have to tell people that it works with any version between 2.4.3 and 2.6.9, not including any 2.5.x release. can't-we-just-call-them-all-bruce-and-avoid-the-confusion?-ly y'rs, -Barry From tim.one@comcast.net Mon Apr 8 22:09:48 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 08 Apr 2002 17:09:48 -0400 Subject: [Python-Dev] 2.2.1 release mechanics In-Reply-To: <15538.1011.400324.537358@grendel.zope.com> Message-ID: [Michael Hudson] >> April 10 ~1200 GMT: release ends >> >> By now, F, T & J have done their bits, uploaded >> files to creosote and sf (or pointed me to where I >> can get them), etc. [Fred] > Are we still on target for this? I expect so, but it's gotten confused. Since I got up today , + I'm building a 2.1.3 Windows installer too. + I believe we're no longer uploading anything to SF, although I'm not sure Michael and Anthony both believe that. + A very recent msg from Guido said we were releasing 2.2.1 today too, which is ahead of Michael's schedule (in response to the 2.1.3 Windows semi-surprise, I had intended to push 2.2.1 Windows off to Tuesday night). From aahz@pythoncraft.com Mon Apr 8 22:15:06 2002 From: aahz@pythoncraft.com (Aahz) Date: Mon, 8 Apr 2002 17:15:06 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15537.17577.734317.166647@anthem.wooz.org> References: <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> <0204080832260F.05787@arthur> <15537.17577.734317.166647@anthem.wooz.org> Message-ID: <20020408211505.GA562@panix.com> On Mon, Apr 08, 2002, Barry A. Warsaw wrote: > > Part of the problem is implementation details (i.e. avoiding the use > of Python 2.2 features in standalone, but not in the Python cvs > tree). Part of it is due to the nature of distutils package layout > (i.e. you need to put setup.py, README, etc. someplace, and distutils > seems to strongly discourage it being in th epackage). Part of it's > also that I feel I have more leeway to change things in the standalone > package. > > But at some point it will stop making sense to distribute a separate > package. Hrm. But until it's no longer a separate package, why use 2.2-specific features in the builtin package? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From aleax@aleax.it Mon Apr 8 22:19:38 2002 From: aleax@aleax.it (Alex Martelli) Date: Mon, 8 Apr 2002 23:19:38 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15538.1515.580133.785503@12-248-41-177.client.attbi.com> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <15537.55112.191870.654238@anthem.wooz.org> <15538.1515.580133.785503@12-248-41-177.client.attbi.com> Message-ID: <0204082319380C.14931@arthur> On Monday 08 April 2002 23:04, Skip Montanaro wrote: ... > requires that sort of coupling, Guido simply calls in a favor from one of > his Italian cousins in the Soprano or Corleone families (my apologies for > the crude American Mafia-style references, Alex), and they take care of Thats'a fin-o, cumpa', i'm-a gonna handle those cases meself... Alex From jeremy@zope.com Mon Apr 8 22:23:11 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 8 Apr 2002 17:23:11 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15538.1515.580133.785503@12-248-41-177.client.attbi.com> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> <15538.1515.580133.785503@12-248-41-177.client.attbi.com> Message-ID: <15538.2623.323978.125527@slothrop.zope.com> It seems like adopting a Linux-style development branch makes lots of extra work and doesn't buy Python much extra testing or stability. What you're calling experimental releases, we currently call cvs checkout :-). I'm happy to keep truly experimental stuff in CVS between releases and aim for stability with each 2.x / minor release. The more releases we do, the more time wasted on packaging of the releases and building docs and installers. We make bug reporting more complicated, because we need to check what CVS version corresponds to 2.3.17. And, as Barry just noted, we lead users to make complicated statements like: "This works with Python versions 2.4.3-2.4.19 and 2.6.0 on up but not 2.5.0-2.5.18." Jeremy From barry@zope.com Mon Apr 8 22:28:53 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Apr 2002 17:28:53 -0400 Subject: [Python-Dev] Re: Stability and change References: <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> <0204080832260F.05787@arthur> <15537.17577.734317.166647@anthem.wooz.org> <20020408211505.GA562@panix.com> Message-ID: <15538.2965.605026.409621@anthem.wooz.org> >>>>> "A" == Aahz writes: A> Hrm. But until it's no longer a separate package, why use A> 2.2-specific features in the builtin package? Because it's convenient, and the semantics are subtly different (and IMO suboptimal) without them. E.g. the builtin version uses a generator in a few places, while the standalone version uses an iterator. The generator version is much more intuitive, but to write cross-version code that uses that interface you need to do something like call list() on a return value. -Barry From skip@pobox.com Mon Apr 8 22:31:04 2002 From: skip@pobox.com (Skip Montanaro) Date: Mon, 8 Apr 2002 16:31:04 -0500 Subject: [Python-Dev] A dizzying stream of checkins In-Reply-To: <200204082046.g38KkSp14849@pcp742651pcs.reston01.va.comcast.net> References: <15537.64937.745224.846065@12-248-41-177.client.attbi.com> <200204082046.g38KkSp14849@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15538.3096.708879.527297@12-248-41-177.client.attbi.com> Guido> Zope Corp uses a superior post-commit action script that seems to Guido> be able to send email to different lists depending to the tag; Guido> maybe we can add that feature to our commit script? Yeah, that would be nice. Does it allow people to subscribe to particular branches or something? I trust the point release managers, so for my needs maybe I can just vector all python-checkins mail matching "Tag: .*maint" into a separate folder to keep it separate from checkins on the main trunk. Skip From skip@pobox.com Mon Apr 8 22:37:34 2002 From: skip@pobox.com (Skip Montanaro) Date: Mon, 8 Apr 2002 16:37:34 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <02040822525808.14931@arthur> References: <200204081734.g38HYwb13124@pcp742651pcs.reston01.va.comcast.net> <15538.334.106174.798218@12-248-41-177.client.attbi.com> <02040822525808.14931@arthur> Message-ID: <15538.3486.710153.139156@12-248-41-177.client.attbi.com> Guido> I think version dependency management is subject to the Logajan Guido> Guido> paradox. :-( Skip> Can you state that succintly for us? :-) Alex> "Python keeps being maintained yet it *never* happens that release Alex> X (for any X and Y) can run some code that release Y cannot run". Alex> It's unachievable (as well as unreasonable) and thus I don't think Alex> it should be considered. The only consideration I was giving it was to maybe submit it to the global fortune cookie file... ;-) Skip From guido@python.org Mon Apr 8 22:41:51 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 17:41:51 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 23:01:00 +0200." <0204082301000A.14931@arthur> References: <02040822525808.14931@arthur> <200204082100.g38L0ol15046@pcp742651pcs.reston01.va.comcast.net> <0204082301000A.14931@arthur> Message-ID: <200204082141.g38Lfp015226@pcp742651pcs.reston01.va.comcast.net> > > Tell that to the people who write L.append(1, 2, 3) despite that it > > was never documented. :-( > > I do -- if and when I see that (haven't seen it in ages). Technical > managers at development shops know about stuff "accidentally > working" and are more tolerant of bug-fixes against that than of > other backwards-compatibility breaks, in my experience. Hm. This was widely complained about, probably because many examples (including in books) got it wrong. Maybe technical manager are tolerant of that stuff, but not if it means they have to fix their won code. :-( --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 8 22:46:24 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 17:46:24 -0400 Subject: [Python-Dev] Pondering some changes to python.c... In-Reply-To: Your message of "Mon, 08 Apr 2002 15:05:23 MDT." <20020408150523.P11182@tummy.com> References: <20020407042038.O16962@tummy.com> <3CB155D1.422AD8C3@lemburg.com> <20020408142523.T16962@tummy.com> <200204082051.g38Kpv314922@pcp742651pcs.reston01.va.comcast.net> <20020408150523.P11182@tummy.com> Message-ID: <200204082146.g38LkOF15266@pcp742651pcs.reston01.va.comcast.net> > On Mon, Apr 08, 2002 at 04:51:57PM -0400, Guido van Rossum wrote: > >When I run a Python script that has a #! directly, ps doesn't show me > >the "python" part -- only the script name. This is in Linux (Red Hat > >7.2). Maybe it's different for you? [Sean] > I was told at one point that "#!/usr/bin/env python" was the preferred > she-bang incantation. However, calling it directly on my 7.2 box seems to > show Python in either case: > > guin:tmp$ printf '#!/usr/bin/python\nimport time; time.sleep(5)' >testscript > guin:tmp$ ./testscript & ps -awlx | grep testscript > 000 500 8297 6741 10 0 2256 1192 do_sel S pts/0 0:00 /usr/bin/python ./testscript > guin:tmp$ printf '#!/usr/bin/env python\nimport time; time.sleep(5)' >testscript > guin:tmp$ ./testscript & ps -awlx | grep testscript > 000 500 8300 6741 9 0 2256 1192 do_sel S pts/0 0:00 python ./testscript Alas, yes. Default ps shows the script name, but ps -f does indeed show the Python interpreter. > >I'm not sure I understand. If you have the argc/argv/environ set, why > >do you need to know &argc and &argv? > > On Linux and a few other systems, setting the ps string requires re-writing > the block of memory starting at argv[0]. If the data in envp is > allocated next to argv, you can also make use of that (so that you can > write strings longer than "python testscript" in length). > > The typical way of doing this involves first making a copy of argv and envp > for the process to continue using, because otherwise argv suddenly becomes > something entirely different than what it originally was. > > So, we need to know &argv so that we can hand back the new location. Sigh. The more I hear about this, the more I think, too ugly, this doesn't belong in Python. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Mon Apr 8 22:50:05 2002 From: skip@pobox.com (Skip Montanaro) Date: Mon, 8 Apr 2002 16:50:05 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15538.1632.252404.964819@anthem.wooz.org> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> <15538.1515.580133.785503@12-248-41-177.client.attbi.com> <15538.1632.252404.964819@anthem.wooz.org> Message-ID: <15538.4237.873211.294420@12-248-41-177.client.attbi.com> BAW> Okay, so now I tell have to tell people that it works with any BAW> version between 2.4.3 and 2.6.9, not including any 2.5.x release. Over time people should become aware that odd-numbered minor releases are always development releases and are not to be relied upon. Sure, you can't say it works for 2.4.3 through 2.6.9 but you can say It works for 2.4.x for x >= 3 and for 2.6.x for x <= 9. And during the initial three years of turmoil after a change to such a dual branch structure is made you'd probably be wise to add No support for any development branch releases (e.g. 2.N.x, where N is odd) is implied. Skip From guido@python.org Mon Apr 8 22:52:40 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 17:52:40 -0400 Subject: [Python-Dev] 2.2.1 release mechanics In-Reply-To: Your message of "Mon, 08 Apr 2002 17:09:48 EDT." References: Message-ID: <200204082152.g38LqeS15350@pcp742651pcs.reston01.va.comcast.net> > I expect so, but it's gotten confused. Since I got up today , > > + I'm building a 2.1.3 Windows installer too. Which you offered to do earlier this weekend. > + I believe we're no longer uploading anything to SF, although I'm > not sure Michael and Anthony both believe that. AFAIK they were cc'ed on the msgs where that was decided. > + A very recent msg from Guido said we were releasing 2.2.1 today > too, which is ahead of Michael's schedule (in response to the > 2.1.3 Windows semi-surprise, I had intended to push 2.2.1 Windows > off to Tuesday night). Fine with me. I didn't mean the reference to "tonight" as a directive to the release manager, it was what *I* believed, sorry if I was wrong. This is Michael's call. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 8 23:00:58 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 18:00:58 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 17:23:11 EDT." <15538.2623.323978.125527@slothrop.zope.com> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> <15538.1515.580133.785503@12-248-41-177.client.attbi.com> <15538.2623.323978.125527@slothrop.zope.com> Message-ID: <200204082200.g38M0wx15409@pcp742651pcs.reston01.va.comcast.net> > It seems like adopting a Linux-style development branch makes lots of > extra work and doesn't buy Python much extra testing or stability. I'm not at all convinced that we should do this, but I don't think the work is that much. We've got long-lived branches for the 2.1 and 2.2 maintenance releases already. > What you're calling experimental releases, we currently call cvs > checkout :-). I'm happy to keep truly experimental stuff in CVS > between releases and aim for stability with each 2.x / minor release. If I believed there was a way to get more people to experiment with fresh code, I'd do it. I had thought that CVS is too high of a barrier, but I think we probably get as much from testers who are running CVS as we get from testers who are downloading alpha and beta releases. > The more releases we do, the more time wasted on packaging of the > releases and building docs and installers. One of the suggestions on the table is to seriously streamline the work we do for a "release" -- perhaps with the exception of "major releases" (which are really "minor" releases: 2.1, 2.2, 2.3). In particular, we would *not* build a Doc release and a Windows installer for each micro release, nor a Mac release. > We make bug reporting more complicated, because we need to check > what CVS version corresponds to 2.3.17. Don't you know how to use CVS or something? We have tags for each release -- if there's a question about this, it's always easily answered. > And, as Barry just noted, we lead users to make complicated > statements like: "This works with Python versions 2.4.3-2.4.19 and > 2.6.0 on up but not 2.5.0-2.5.18." Apparently that's not a problem for Linux. I'd say that since nobody except bleeding edge developers uses a 2.5.x, all you'd have to say would be "this works on 2.4.3 and newer stable releases". --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Mon Apr 8 22:58:27 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Apr 2002 17:58:27 -0400 Subject: [Python-Dev] Re: Stability and change References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> <15538.1515.580133.785503@12-248-41-177.client.attbi.com> <15538.1632.252404.964819@anthem.wooz.org> <15538.4237.873211.294420@12-248-41-177.client.attbi.com> Message-ID: <15538.4739.536592.90221@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: BAW> Okay, so now I tell have to tell people that it works with BAW> any version between 2.4.3 and 2.6.9, not including any 2.5.x BAW> release. SM> Over time people should become aware that odd-numbered minor SM> releases are always development releases and are not to be SM> relied upon. Sure, you can't say it works for 2.4.3 through SM> 2.6.9 but you can say SM> It works for 2.4.x for x >= 3 and for 2.6.x for x <= 9. SM> And during the initial three years of turmoil after a change SM> to such a dual branch structure is made you'd probably be wise SM> to add SM> No support for any development branch releases SM> (e.g. 2.N.x, where N is odd) is implied. IMHO it will still confuse a lot of people. I don't think it's the same as the Linux kernel because I suspect that people upgrade their kernels a /lot/ less often then they'll upgrade their Python version. -Barry From geoff-pdev@gerrietts.net Mon Apr 8 22:49:46 2002 From: geoff-pdev@gerrietts.net (Geoff Gerrietts) Date: Mon, 8 Apr 2002 14:49:46 -0700 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15538.3486.710153.139156@12-248-41-177.client.attbi.com> References: <200204081734.g38HYwb13124@pcp742651pcs.reston01.va.comcast.net> <15538.334.106174.798218@12-248-41-177.client.attbi.com> <02040822525808.14931@arthur> <15538.3486.710153.139156@12-248-41-177.client.attbi.com> Message-ID: <20020408214946.GD10251@isis.gerrietts.net> Quoting Skip Montanaro (skip@pobox.com): > Guido> I think version dependency management is subject to the Logajan > Guido> Guido> paradox. :-( > > Skip> Can you state that succintly for us? :-) > > Alex> "Python keeps being maintained yet it *never* happens that release > Alex> X (for any X and Y) can run some code that release Y cannot run". > > Alex> It's unachievable (as well as unreasonable) and thus I don't think > Alex> it should be considered. > > The only consideration I was giving it was to maybe submit it to the global > fortune cookie file... ;-) I think that while I agree that there's a paradox there, what we're seeing is Logajan failing to express himself, not Logajan asking for a dead language. I think Logajan's idea was that backwards-compatibility breaks were bad, where you (for instance) write code for Tkinter under 1.5.2 and in 2.whatever it breaks because of Unicode changes. But it's worse when you can't write that code such that it works in 2.whatever and also works in 1.5.2 without resorting to testing version numbers and special-casing. Maybe that translates to the same thing for you, but for me, it's a very different thing. Further, I think that the issue would be quashed if it were simple to work around, rather than complex. If you could set a global variable in sys or tkinter or __main__ and be assured that you could write cross-version compatible code, that would be much preferable to having two write two different sets of modules, one that works under 1.5 and one that works under 2.0, and one that works under 2.1. I could be mistken, but I think that's what I've heard. Thanks, --G. -- Geoff Gerrietts "Democracy is a form of government that substitutes geoff at gerrietts dot net election by the incompetant many for appointment http://www.gerrietts.net/ by the corrupt few." --George Bernard Shaw From geoff-pdev@gerrietts.net Mon Apr 8 22:51:00 2002 From: geoff-pdev@gerrietts.net (Geoff Gerrietts) Date: Mon, 8 Apr 2002 14:51:00 -0700 Subject: [Python-Dev] Re: Stability and change Message-ID: <20020408215100.GE10251@isis.gerrietts.net> Quoting Guido van Rossum (guido@python.org): > Before 1.5.2, we *also* hit 6 months with some regularity. The only > irregularity was really the 18 month gap between 1.5.2 and 2.0. We're > seeing the gap slowly increase, to maybe 8 months for 2.3, but I > expect that 6-8 months is probably what we'll continue to see. Which is reasonable, as I've tried to state. But an explicit statement "We aim for a six-month development cycle with one to two months of slop" is better than implicitly gathering as much from a review of documentation release dates (the only handy way I've found to date releases, short of knowing them off your head). > > - Be predictable. If we're putting off pain until 3.0, make a guess > > when we're going to need to deal with that pain, so we can start > > preparing. If 3.0 is 9 months off, that's a lot scarier and more > > concerning than if it's 2 years off. > > 3.0 will always be 2 years off. :-) At least for now, it really is. Again, reasonable, but not explicitly announced (or if it has been, it's not vocally announced). I'm not at this point looking for comfort, having more or less made my bed. It's nice to find warm sheets, but I'd be stuck with whatever I found. Others are looking for that kind of comfort, though. While this list might have a good idea that "deferred for Python 3.0" means "in 2 years, you'll need to care", I feel like the statement contains a great deal of ambiguity for those who aren't. > > - Be explicit about the process. If the process is to stick a fork in > > the ham and call it "stable" just before moving on to the next minor > > release, then be explicit about that process. > > We are already pretty explicit; there are several PEPs about the > process. I think that considering a PEP as explicit explanation of the process is a lot like expecting cvs log messages to replace Andrew Kuchling's "What's New" summaries. Maybe what I mean is not explicit. Maybe I should be hunting for another word -- how about "forthright"? That's still not ideal, but the idea is that everyone should know; it should be shouted from the rooftops. I think that PEPs are a great tool when they're not fuel for flames and invective. They're not documentation, though -- they're project proposals. The audience you reach with a PEP is entirely different from the audience I'm suggesting you need to reach with this message. > It depends on the bugfix. Some modules and subsystems are being > refactored in order to provide new features, and then bugfixes won't > port back. Other modules and subsystems are very stable (some could > be called stagnant :-) and will have no problem getting bugfixes > backported. I think maybe I need to be more concrete about this. The last /appearance/ of stability -- the last time something looked like a fixed target -- was 1.5.2, for whatever reason. Lots of people continue to target 1.5.2, and you can cite a bajillion reasons why they might. At first occurance of any problem that may or may not be an interpreter bug or a standard library defect, the advice given is to upgrade -- reasonable advice! But that's the suggestion, often without including a recommendation for which of the newer releases to target. 1.5.2 is obsolescent, and it would be very useful for someone who's aiming at 2.2 to know how long his or her company could expect to continue using 2.2 before the first answer to any new problem s/he might experience was "upgrade". This is a hard assurance to come by, and a hard promise to make in a community as dynamic and "scratch my own itch" as python's (or any open source project, for that matter). Nonetheless, THIS is the stability benchmark that most corporate clients want a sense of. > Yes, maybe it can be made to disappear. Does that mean we need to > throw the reactionaries on c.l.py a bone? Is this really all > politics? In that case I'm out of here. I think it's mostly communication, not politics. The two can be difficult to tell apart, but the former tends to involve telling the truth, a lot; the latter tends to involve telling fibs, a lot. :) To more seriously address your concern, my belief is that people who fear change, fear ambiguity. Managing that fear involves reducing the ambiguity where you can, and demarking the limits of the ambiguity where it can't be removed. The best tool for both tasks is explicit, clear communication. I have tried to identify the places where I see the largest ambiguities looming. My belief is that if these ambiguities are controlled by directly addressing them, some portion of the problems will go away. I am told that there's a lot of psychological evidence to support this belief, but I haven't done the lit search and am not especially inclined to do so without a compelling reason. :) -- Geoff Gerrietts "Me and my homies, we tag O.D.." --Unknown grafitti artist at a party From jeremy@alum.mit.edu Mon Apr 8 19:12:39 2002 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Mon, 08 Apr 2002 14:12:39 -0400 (EDT) Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081225.g38CPNn10186@mbuna.arbhome.com.au> References: <2m7kni5rvk.fsf@starship.python.net> <200204081225.g38CPNn10186@mbuna.arbhome.com.au> Message-ID: <15537.56727.412033.844839@walden.zope.com> >>>>> "AB" == Anthony Baxter writes: >>>> Michael Hudson wrote >> I'm only scanning this huge thread, so I may be out of >> context/repetitive. AB> You're not the only one :) I'm actually reading the whole thread, but I've only read 45% of the messages that arrived before 2:40 p.m. :-) AB> The key thing, I think, is to keep on top of the backporting. If AB> it slips, it's an absolute monster to catch up. If we are going to change anything about the way we work, this strikes me as the most likely candidate. If we are going to continue to maintain 2.1, then a developer who checks in a fix for some bug on the trunk should also fix it for 2.2 and 2.1 if possible. Basically, we need unambigious rules about when we'll stop doing micro releases from a particular branch. I think we should maintaince on 2.1 and 2.2, largely because 2.2 has so much experimental stuff. Once we get to 2.3, we can re-evaluate the situation and decide whether we want to continue maintaining 2.1. Jeremy P.S. >> > I have a real problem when a group of people demands something >> > but won't volunteer anything to make it happen except their >> > complaints; >> Indeed; they can fuck off, in my book. AB> Absolutely. I believe the phrase "Go Stick Your Head In A Pig" AB> is also appropriate. >> > Isn't this dangerous? I find it funny that Michael's asked the question before you wrote your email. I'm glad the time machine gets used for jokes now and again. From aahz@pythoncraft.com Mon Apr 8 23:01:20 2002 From: aahz@pythoncraft.com (Aahz) Date: Mon, 8 Apr 2002 18:01:20 -0400 Subject: [Python-Dev] PEP Politics (was Re: Re: Stability and change) In-Reply-To: <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> References: <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <15537.11319.576212.846144@anthem.wooz.org> <15537.15490.631372.551394@12-248-41-177.client.attbi.com> <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020408220120.GB562@panix.com> On Mon, Apr 08, 2002, Guido van Rossum wrote: > >> Guido> Maybe I should PEP it first? :-) >> >> You've got a smiley there, > > Because any PEP I write appears to mostly raise controversy once > brought up on c.l.py. >From what I've seen, much of the controversy revolves around what might be described as infelicitous phrasing. Discussion on python-dev tends to focus on the actual issues, but that changes once a PEP goes "into the wild". Perhaps right before you wish to release a PEP to c.l.py, you should ask for an editing run specifically to focus on the political side. This will not avoid the mud from certain quarters, of course, but they probably should be ignored. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From aahz@pythoncraft.com Mon Apr 8 23:14:14 2002 From: aahz@pythoncraft.com (Aahz) Date: Mon, 8 Apr 2002 18:14:14 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15538.2965.605026.409621@anthem.wooz.org> References: <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> <0204080832260F.05787@arthur> <15537.17577.734317.166647@anthem.wooz.org> <20020408211505.GA562@panix.com> <15538.2965.605026.409621@anthem.wooz.org> Message-ID: <20020408221414.GC562@panix.com> On Mon, Apr 08, 2002, Barry A. Warsaw wrote: > > >>>>> "A" == Aahz writes: > > A> Hrm. But until it's no longer a separate package, why use > A> 2.2-specific features in the builtin package? > > Because it's convenient, and the semantics are subtly different (and > IMO suboptimal) without them. E.g. the builtin version uses a > generator in a few places, while the standalone version uses an > iterator. The generator version is much more intuitive, but to write > cross-version code that uses that interface you need to do something > like call list() on a return value. Well, sure. But you have to do that suboptimal stuff *anyway* in order to maintain the separate package. How is the overall development process more convenient to maintain two codebases? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From barry@zope.com Mon Apr 8 23:18:54 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Apr 2002 18:18:54 -0400 Subject: [Python-Dev] Re: Stability and change References: <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> <0204080832260F.05787@arthur> <15537.17577.734317.166647@anthem.wooz.org> <20020408211505.GA562@panix.com> <15538.2965.605026.409621@anthem.wooz.org> <20020408221414.GC562@panix.com> Message-ID: <15538.5966.52028.301619@anthem.wooz.org> >>>>> "A" == Aahz writes: A> Well, sure. But you have to do that suboptimal stuff *anyway* A> in order to maintain the separate package. How is the overall A> development process more convenient to maintain two codebases? It isn't! And now that I've announced email 2.0, I'm going to merge it all into Python 2.3 cvs (but not the suboptimal implementation. ;). I suspect updating the docs will be the most work. -Barry From hughett@mercur.uphs.upenn.edu Mon Apr 8 23:19:45 2002 From: hughett@mercur.uphs.upenn.edu (Paul Hughett) Date: Mon, 8 Apr 2002 18:19:45 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081607.g38G7iD12531@pcp742651pcs.reston01.va.comcast.net> (message from Guido van Rossum on Mon, 08 Apr 2002 12:07:44 -0400) References: <200204081607.g38G7iD12531@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204082219.g38MJj614704@mercur.uphs.upenn.edu> As an extension developer, I see considerable advantages to having separate stable and experimental tracks, a la Linux. The key distinction, as I see it, is that the stable track has most of the bugs fixed, is recommended for production use (though not perfect), and that any code that works with stable version N will almost certainly work with version N+1, but it doesn't have all the latest and greatest features. The experimental track, on the other hand, has all the latest and greatest, but they might not work and the details might change from version to version. I then get several advantages from sticking to the stable versions: (1) Any bugs I find are probably in my extension rather than in the Python implementation. (2) The bugs that I find are probably consistent across most of the minor versions. (3) I only have to write for and support half as many different major versions. Given that the implementations are more stable, my workload is cut by a factor of more than two. (4) The documentation for the stable versions is (probably) better quality, so I have to spend less time experimenting to get things to work as I want. Given that we distinguish stable and experimental versions (which I regard as an obvious +1), we have the problem of labelling them and I think the Linux system is well suited (but I'm still willing to listen to alternatives): It's well-established and known to a large part of our target audience. It's pretty easy to determine from the version number alone whether it's stable (minor version even) or experimental odd); I wouldn't want to try to remember that--say--versions 2.2.1 through 2.2.19 are experimental while 2.2.20 and up are stable. Paul Hughett From nas@python.ca Mon Apr 8 23:24:04 2002 From: nas@python.ca (Neil Schemenauer) Date: Mon, 8 Apr 2002 15:24:04 -0700 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081842.g38Ig4k13782@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Mon, Apr 08, 2002 at 02:42:04PM -0400 References: <20020408180043.GA2726@crystal.mems-exchange.org> <20020408113014.A24764@glacier.arctrix.com> <200204081842.g38Ig4k13782@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020408152404.B24764@glacier.arctrix.com> Guido van Rossum wrote: > I don't know how they do it. Can you explain? It's explained in this document: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/current-stable.html From tim.one@comcast.net Mon Apr 8 23:28:57 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 08 Apr 2002 18:28:57 -0400 Subject: [Python-Dev] 2.2.1 release mechanics In-Reply-To: <200204082152.g38LqeS15350@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Tim] >> I expect so, but it's gotten confused. Since I got up today , >> >> + I'm building a 2.1.3 Windows installer too. [Guido] > Which you offered to do earlier this weekend. Sure, but it was left undecided then -- I offer to do a lot of things it turns out nobody wants . >> + I believe we're no longer uploading anything to SF, although I'm >> not sure Michael and Anthony both believe that. > AFAIK they were cc'ed on the msgs where that was decided. Then perhaps I wasn't. I only deduced it indirectly from one piece of email that *suggested* it might be the case, buried in a layer or two of quoting. So I believe it's prudent to make sure everyone who was supposed to get that email actually did (esp. since I don't believe I did, unless it's hiding in one of the 100+ new "stability" emails I haven't had time to open yet). >> + A very recent msg from Guido said we were releasing 2.2.1 today >> too, which is ahead of Michael's schedule (in response to the >> 2.1.3 Windows semi-surprise, I had intended to push 2.2.1 Windows >> off to Tuesday night). > Fine with me. I didn't mean the reference to "tonight" as a directive > to the release manager, it was what *I* believed, sorry if I was > wrong. This is Michael's call. Cool. Michael has stuck to his release schedule since announcing it, and I see no reason we can't meet it. From guido@python.org Mon Apr 8 23:34:45 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 18:34:45 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 14:51:00 PDT." <20020408215100.GE10251@isis.gerrietts.net> References: <20020408215100.GE10251@isis.gerrietts.net> Message-ID: <200204082234.g38MYjm15638@pcp742651pcs.reston01.va.comcast.net> > But an explicit statement "We aim for a six-month development cycle > with one to two months of slop" is better than implicitly gathering > as much from a review of documentation release dates (the only handy > way I've found to date releases, short of knowing them off your > head). How about the dates mentioned on http://www.python.org/download/ ? Or PEP 283 (the 2.3 release schedule)? [on 3.0] > Again, reasonable, but not explicitly announced (or if it has been, > it's not vocally announced). I think you're being unreasonable. Try getting a straight answer from Microsoft on when they'll release the successor to XP. Or Linus when kernel 2.6 will be out. > I'm not at this point looking for comfort, having more or less made > my bed. It's nice to find warm sheets, but I'd be stuck with > whatever I found. Others are looking for that kind of comfort, > though. While this list might have a good idea that "deferred for > Python 3.0" means "in 2 years, you'll need to care", I feel like the > statement contains a great deal of ambiguity for those who aren't. It's a big project. It'll probably take longer than expected. We can't predict the future. That's the only answer I can give you without consulting a lawyer for the right amount of weasel words. > > We are already pretty explicit; there are several PEPs about the > > process. > > I think that considering a PEP as explicit explanation of the process > is a lot like expecting cvs log messages to replace Andrew Kuchling's > "What's New" summaries. Now you're being unreasonable again. You don't expect us to do press releases about this, do you? > Maybe what I mean is not explicit. Maybe I should be hunting for > another word -- how about "forthright"? That's still not ideal, but > the idea is that everyone should know; it should be shouted from the > rooftops. Can't do that. All we have to shout from the rooftops is "Python 3000 is coming". But not when, or what. It's not that we don't want you to know. We don't know it ourselves yet. > I think that PEPs are a great tool when they're not fuel for flames > and invective. They're not documentation, though -- they're project > proposals. The audience you reach with a PEP is entirely different > from the audience I'm suggesting you need to reach with this message. PEPs also serve as documentation. Just like RFCs, they serve as a specification. > > It depends on the bugfix. Some modules and subsystems are being > > refactored in order to provide new features, and then bugfixes won't > > port back. Other modules and subsystems are very stable (some could > > be called stagnant :-) and will have no problem getting bugfixes > > backported. > > I think maybe I need to be more concrete about this. The last > /appearance/ of stability -- the last time something looked like a > fixed target -- was 1.5.2, for whatever reason. Lots of people > continue to target 1.5.2, and you can cite a bajillion reasons why > they might. And that's fine with me. I still happily use Windows 98 (I hope that's fine with Bill Gates too :-). > At first occurance of any problem that may or may not be an > interpreter bug or a standard library defect, the advice given is to > upgrade -- reasonable advice! But that's the suggestion, often without > including a recommendation for which of the newer releases to target. > > 1.5.2 is obsolescent, and it would be very useful for someone who's > aiming at 2.2 to know how long his or her company could expect to > continue using 2.2 before the first answer to any new problem s/he > might experience was "upgrade". Again, that's impossible to explain without knowing what kind of use they make of Python 2.2, what kind of developers they have, and so on. I think this is to a large extent something that everybody has to decide for themselves. If you have no idea, you might want to ask someone else who is in a similar user category as yourself. But the language developers can't know whether 1.5.2, 2.1 or 2.3 is best for you. > This is a hard assurance to come by, and a hard promise to make in a > community as dynamic and "scratch my own itch" as python's (or any > open source project, for that matter). Nonetheless, THIS is the > stability benchmark that most corporate clients want a sense of. Too bad. Until they start offering me money, I'm not sure I should care. If someone says "change X or else I will stop using your language", my response is usually "your loss". If someone asks "how much would it cost to change X" I might listen. > > Yes, maybe it can be made to disappear. Does that mean we need to > > throw the reactionaries on c.l.py a bone? Is this really all > > politics? In that case I'm out of here. > > I think it's mostly communication, not politics. The two can be > difficult to tell apart, but the former tends to involve telling the > truth, a lot; the latter tends to involve telling fibs, a lot. :) When there's a vocal minority in the newsgroup spreading misinformation about Python's stability (e.g. by continuously whining about the same issues), that's politics too. > To more seriously address your concern, my belief is that people who > fear change, fear ambiguity. Managing that fear involves reducing the > ambiguity where you can, and demarking the limits of the ambiguity > where it can't be removed. The best tool for both tasks is explicit, > clear communication. I have tried to identify the places where I see > the largest ambiguities looming. My belief is that if these > ambiguities are controlled by directly addressing them, some portion > of the problems will go away. I don't think you can have what you're asking for here -- I have no unambiguous answers to your questions beyond "I'm doing the best I can" (or else I wouldn't be spending this entire day reading and responding to email on this topic). I think that usually, rather than expecting unambiguous answers (which may simply be bluff), you need to listen to the grapevine. Python's grapevine used to be pretty reliable. Unfortunately there's a lot of noise lately. But I can't really think of anything I could say that would be honest *and* take away the ambiguity. So be it. > I am told that there's a lot of psychological evidence to support this > belief, but I haven't done the lit search and am not especially > inclined to do so without a compelling reason. :) I'm trying to be as direct as I can here. I have nothing to hide. The answer simply isn't as clear as you'd like it to be. Tough. --Guido van Rossum (home page: http://www.python.org/~guido/) From hughett@mercur.uphs.upenn.edu Mon Apr 8 23:31:23 2002 From: hughett@mercur.uphs.upenn.edu (Paul Hughett) Date: Mon, 8 Apr 2002 18:31:23 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081757.g38HvVs13378@pcp742651pcs.reston01.va.comcast.net> (message from Guido van Rossum on Mon, 08 Apr 2002 13:57:31 -0400) References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081738.g38HcvF13190@pcp742651pcs.reston01.va.comcast.net> <200204081757.g38HvVs13378@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204082231.g38MVNO14731@mercur.uphs.upenn.edu> > We could jump the micro release number to 100. [to mark the experimental to > stable transition] -0 Why invent a new convention when a good one already exists? How do the development groups other than Linux make this distinction? Paul Hughett From guido@python.org Mon Apr 8 23:35:57 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 18:35:57 -0400 Subject: [Python-Dev] Re: PEP Politics (was Re: Re: Stability and change) In-Reply-To: Your message of "Mon, 08 Apr 2002 18:01:20 EDT." <20020408220120.GB562@panix.com> References: <200204061548.g36FmJl28581@pcp742651pcs.reston01.va.comcast.net> <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <15537.11319.576212.846144@anthem.wooz.org> <15537.15490.631372.551394@12-248-41-177.client.attbi.com> <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> Message-ID: <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> > From what I've seen, much of the controversy revolves around what > might be described as infelicitous phrasing. Discussion on > python-dev tends to focus on the actual issues, but that changes > once a PEP goes "into the wild". Perhaps right before you wish to > release a PEP to c.l.py, you should ask for an editing run > specifically to focus on the political side. > > This will not avoid the mud from certain quarters, of course, but > they probably should be ignored. I very much doubt it. If the language is too slick, we'll get complaints about a "marketing PEP" (see PEP 287's fate). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 8 23:36:40 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 18:36:40 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 17:58:27 EDT." <15538.4739.536592.90221@anthem.wooz.org> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> <15538.1515.580133.785503@12-248-41-177.client.attbi.com> <15538.1632.252404.964819@anthem.wooz.org> <15538.4237.873211.294420@12-248-41-177.client.attbi.com> <15538.4739.536592.90221@anthem.wooz.org> Message-ID: <200204082236.g38Maeg15671@pcp742651pcs.reston01.va.comcast.net> > SM> No support for any development branch releases > SM> (e.g. 2.N.x, where N is odd) is implied. [Barry] > IMHO it will still confuse a lot of people. I don't think it's the > same as the Linux kernel because I suspect that people upgrade their > kernels a /lot/ less often then they'll upgrade their Python version. Yup. Most people don't even know what kernel (or what gcc, etc.) they have -- all they know is that they installed Red Hat 7.2. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Mon Apr 8 23:37:49 2002 From: skip@pobox.com (Skip Montanaro) Date: Mon, 8 Apr 2002 17:37:49 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15538.2623.323978.125527@slothrop.zope.com> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> <15538.1515.580133.785503@12-248-41-177.client.attbi.com> <15538.2623.323978.125527@slothrop.zope.com> Message-ID: <15538.7101.351365.784668@12-248-41-177.client.attbi.com> Jeremy> It seems like adopting a Linux-style development branch makes Jeremy> lots of extra work and doesn't buy Python much extra testing or Jeremy> stability. What you're calling experimental releases, we Jeremy> currently call cvs checkout :-). I'm happy to keep truly Jeremy> experimental stuff in CVS between releases and aim for stability Jeremy> with each 2.x / minor release. Perhaps you're right. The problem with simply dealing off the top of the CVS deck is that it provides no way to conclusively state that the current CVS head is either more functionally complete or more bug-free than an earlier CVS head. Hopefully releases will be must smaller efforts than they currently are. Pulling away from SF for releases will certainly help. If a micro release on the development branch can asymptotically approach * create a baseline named devel-2.N.x * drop a tarfile onto www.python.org * send out an email I think you could probably justify making releases on that branch every month or so. Releases on the stable branch would probably occur less frequently (only when a sufficient number of bug fixes had accumulated or a particularly serious bug fix was applied to the branch) and be a bit more involved (maybe involving creation of a Windows installer). Above and beyond the notion of complexity or technical superiority of different development models, I think you need to realize a positive PR advantage from a change in development models. If you could produce a table like Version Bugs fixed ------- ---------- 2.4.1 47 2.4.2 32 2.4.3 12 2.4.4 3 you would go a long way toward arguing that "stable" branches are really getting more stable. Similarly, such a table on the development branch might help developers decide if the development branch is stable enough to fork. Skip From guido@python.org Mon Apr 8 23:40:01 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 18:40:01 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 14:12:39 EDT." <15537.56727.412033.844839@walden.zope.com> References: <2m7kni5rvk.fsf@starship.python.net> <200204081225.g38CPNn10186@mbuna.arbhome.com.au> <15537.56727.412033.844839@walden.zope.com> Message-ID: <200204082240.g38Me1n15697@pcp742651pcs.reston01.va.comcast.net> [Jeremy] > I'm actually reading the whole thread, but I've only read 45% of the > messages that arrived before 2:40 p.m. :-) Don't respond to anything you see before 6:45, it's all superceded already. :-) > AB> The key thing, I think, is to keep on top of the backporting. If > AB> it slips, it's an absolute monster to catch up. > > If we are going to change anything about the way we work, this strikes > me as the most likely candidate. If we are going to continue to > maintain 2.1, then a developer who checks in a fix for some bug on the > trunk should also fix it for 2.2 and 2.1 if possible. Hm, I think 2.1 may be a lost cause, but we've done this pretty consistently for 2.2, and I strongly encourage to keep doing it. If Michael can no longer do it, I'd like to call for a new volunteer, or (if all else fails) have someone at PythonLabs take over. > Basically, we need unambigious rules about when we'll stop doing micro > releases from a particular branch. I think we should maintaince on > 2.1 and 2.2, largely because 2.2 has so much experimental stuff. > Once we get to 2.3, we can re-evaluate the situation and decide > whether we want to continue maintaining 2.1. I think 2.1 is at the point where we only need to be reactive (e.g. fix reported core dumps). The focus should be to keep 2.2 alive -- sooner or later it will gain a reputation of stability. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@pythoncraft.com Mon Apr 8 23:36:55 2002 From: aahz@pythoncraft.com (Aahz) Date: Mon, 8 Apr 2002 18:36:55 -0400 Subject: [Python-Dev] Developer resources (was Re: Stability and change) In-Reply-To: <15538.2623.323978.125527@slothrop.zope.com> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> <15538.1515.580133.785503@12-248-41-177.client.attbi.com> <15538.2623.323978.125527@slothrop.zope.com> Message-ID: <20020408223655.GD562@panix.com> On Mon, Apr 08, 2002, Jeremy Hylton wrote: > > It seems like adopting a Linux-style development branch makes lots of > extra work and doesn't buy Python much extra testing or stability. > What you're calling experimental releases, we currently call cvs > checkout :-). I'm happy to keep truly experimental stuff in CVS > between releases and aim for stability with each 2.x / minor release. I'm picking this post as a convenient tag for changing the direction of the conversation a bit: I believe that the key issue I've seen mentioned in this thread is the lack of downloads for alpha/beta releases. In turn, that brings up the whole issue of expanding the development and QA resources available to Python as a whole. What I'm seeing in this discussion is an assumption that there's no simple way to expand the available eyeballs. I hope that's wrong. Let's try having a discussion about ways to change that. My experience is that I've been unwilling until recently to participate heavily in development because all I had was a Win98 box. Tim Peters aside, I think most people think of Win98 as a *lousy* platform for development. But even now that I've finally got a Madrake 8.1 box set up, I'm still proceeding more slowly than I'd prefer because I don't want to screw up my box (I've already had many more problems than I expected installing Mandrake). That's aside from little issues like not knowing how to use CVS or my 56k connection or SF bug submissions not working with Lynx. I think I'm fairly typical of the crowd that we'd like to add to the Python resources: reasonably competent technically (but with a *lot* of gaps in knowledge/skills), but also a bit gunshy of pushing limits too quickly. Here are a couple of suggestions: * Create a python-dev-tutor list for people who want to learn how to participate in Python development. * Make the current CVS build available on some semi-public machine like Starship (and by "available", I mean compiled). -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From jeremy@zope.com Mon Apr 8 23:33:55 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 8 Apr 2002 18:33:55 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204082219.g38MJj614704@mercur.uphs.upenn.edu> References: <200204081607.g38G7iD12531@pcp742651pcs.reston01.va.comcast.net> <200204082219.g38MJj614704@mercur.uphs.upenn.edu> Message-ID: <15538.6867.127517.168835@slothrop.zope.com> >>>>> "PH" == Paul Hughett writes: PH> As an extension developer, I see considerable advantages to PH> having separate stable and experimental tracks, a la Linux. Paul, Do you ever do build Python from a cvs checkout? It seems to me that what you call an experimental track, we call a "cvs checkout." What you call a stable releases, we call a release. PH> The key distinction, as I see it, is that the stable track has PH> most of the bugs fixed, is recommended for production use PH> (though not perfect), and that any code that works with stable PH> version N will almost certainly work with version N+1, but it PH> doesn't have all the latest and greatest features. I think all of this was true of Python 2.1 & 2.2 and will be true of 2.3. Sure, there are bugs in 2.1 & 2.2, but we fixed the ones we knew how to fix. I expect that "stable" Linux kernel releases also have bugs, else I wouldn't have a 2.2.12 release on my home machine :-). Jeremy From skip@pobox.com Mon Apr 8 23:43:16 2002 From: skip@pobox.com (Skip Montanaro) Date: Mon, 8 Apr 2002 17:43:16 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15538.4739.536592.90221@anthem.wooz.org> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> <15538.1515.580133.785503@12-248-41-177.client.attbi.com> <15538.1632.252404.964819@anthem.wooz.org> <15538.4237.873211.294420@12-248-41-177.client.attbi.com> <15538.4739.536592.90221@anthem.wooz.org> Message-ID: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> SM> No support for any development branch releases SM> (e.g. 2.N.x, where N is odd) is implied. BAW> IMHO it will still confuse a lot of people. I don't think it's the BAW> same as the Linux kernel because I suspect that people upgrade BAW> their kernels a /lot/ less often then they'll upgrade their Python BAW> version. People (and organizations) who are risk averse will just have to settle for even-numbered micro releases. If, as I suspect, most people get (or will eventually get) Python from running a Windows installer or from their Linux distribution, they will probably be blissfully unaware that odd-numbered minor releases even exist. Windows installers should not be created for development releases and Linux distributions will have no business installing a Python development release in /usr. Skip From barry@zope.com Mon Apr 8 23:41:37 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Apr 2002 18:41:37 -0400 Subject: [Python-Dev] Re: Stability and change References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> <15538.1515.580133.785503@12-248-41-177.client.attbi.com> <15538.1632.252404.964819@anthem.wooz.org> <15538.4237.873211.294420@12-248-41-177.client.attbi.com> <15538.4739.536592.90221@anthem.wooz.org> <200204082236.g38Maeg15671@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15538.7329.491888.577299@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> Yup. Most people don't even know what kernel (or what gcc, GvR> etc.) they have -- all they know is that they installed Red GvR> Hat 7.2. Exactly! -Barry From aahz@pythoncraft.com Mon Apr 8 23:43:59 2002 From: aahz@pythoncraft.com (Aahz) Date: Mon, 8 Apr 2002 18:43:59 -0400 Subject: [Python-Dev] Re: PEP Politics (was Re: Re: Stability and change) In-Reply-To: <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <15537.11319.576212.846144@anthem.wooz.org> <15537.15490.631372.551394@12-248-41-177.client.attbi.com> <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020408224359.GE562@panix.com> On Mon, Apr 08, 2002, Guido van Rossum wrote: > >> From what I've seen, much of the controversy revolves around what >> might be described as infelicitous phrasing. Discussion on >> python-dev tends to focus on the actual issues, but that changes >> once a PEP goes "into the wild". Perhaps right before you wish to >> release a PEP to c.l.py, you should ask for an editing run >> specifically to focus on the political side. >> >> This will not avoid the mud from certain quarters, of course, but >> they probably should be ignored. > > I very much doubt it. If the language is too slick, we'll get > complaints about a "marketing PEP" (see PEP 287's fate). You may be right, but you misunderstood what I was suggesting: I'm simply talking about looking at the PEP from the perspective of Joe Stupid Pythonista, figuring out where the screams are likely to come from, and clarify the language or write pre-emptive rebuttals. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From skip@pobox.com Mon Apr 8 23:48:20 2002 From: skip@pobox.com (Skip Montanaro) Date: Mon, 8 Apr 2002 17:48:20 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15537.56727.412033.844839@walden.zope.com> References: <2m7kni5rvk.fsf@starship.python.net> <200204081225.g38CPNn10186@mbuna.arbhome.com.au> <15537.56727.412033.844839@walden.zope.com> Message-ID: <15538.7732.116435.569021@12-248-41-177.client.attbi.com> Jeremy> Basically, we need unambigious rules about when we'll stop doing Jeremy> micro releases from a particular branch. Agreed. As I've stated in bits and pieces in earlier messages, those rules should involve: * a set of goals for a given development branch * assessment of how completely those goals have been realized * bug fix rate trends after it's been determined the branch is functionally complete. Skip From petrilli@amber.org Mon Apr 8 23:46:46 2002 From: petrilli@amber.org (Christopher Petrilli) Date: Mon, 08 Apr 2002 18:46:46 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15538.4237.873211.294420@12-248-41-177.client.attbi.com> Message-ID: On 4/8/02 5:50 PM, "Skip Montanaro" wrote: > > BAW> Okay, so now I tell have to tell people that it works with any > BAW> version between 2.4.3 and 2.6.9, not including any 2.5.x release. > > Over time people should become aware that odd-numbered minor releases are > always development releases and are not to be relied upon. Sure, you can't > say it works for 2.4.3 through 2.6.9 but you can say > > It works for 2.4.x for x >= 3 and for 2.6.x for x <= 9. > > And during the initial three years of turmoil after a change to such a dual > branch structure is made you'd probably be wise to add > > No support for any development branch releases (e.g. 2.N.x, where N is > odd) is implied. What Barry is trying to say, and I most certainly agree with his most rockiness, is that there shouldn't be oral tradition, it should be obvious, on first inspection. Just because some percentage of the planet that uses Linux understands this numbering scheme (and I'd bet that only 20-30% of the Linux users actually do),doesn't mean it's fare to assume all users would at first blush. The double-branch option is easier, and I know that when the FreeBSD group releases 4.5-RELEASE, it's been tested and solid, and will be supported by bug fixes for some period of time. They have only just recently stopped issue security patches for the 2.x tree. Someone asked how long someone should assume "stability" of a release, or at least viability of a release, and I would say that 2 release cycles is a good number. In other words, if you're on 2.2, when 2.5 comes out, you won't get any more patches or support. One thought that has been in my mind for years, as Guido knows (I whined about it in the 1.3->1.4->1.5 days) is that we are overly conservative in our "major" release information. Perhaps we would be better served from a PR perspective (because in the end, that's all we're talking about) if we were to make 1 "major" release per year, and some minor number after that. That minor number wouldn't exceed 3-4, but there would be some guarantee that within that "major" train, all things were compatible. It's just an idea, but I'm really opposed to magic numbers in determining whether something is a stable or experimental release. We might as well say that only those releases that are prime numbers are in fact stable. That would give us the distinct advantage of longer and longer cycles as we make releases :-) Chris -- | Christopher Petrilli | petrilli@amber.org From aahz@pythoncraft.com Mon Apr 8 23:47:47 2002 From: aahz@pythoncraft.com (Aahz) Date: Mon, 8 Apr 2002 18:47:47 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204082219.g38MJj614704@mercur.uphs.upenn.edu> References: <200204081607.g38G7iD12531@pcp742651pcs.reston01.va.comcast.net> <200204082219.g38MJj614704@mercur.uphs.upenn.edu> Message-ID: <20020408224747.GF562@panix.com> On Mon, Apr 08, 2002, Paul Hughett wrote: > > Given that we distinguish stable and experimental versions (which I > regard as an obvious +1), we have the problem of labelling them and I > think the Linux system is well suited (but I'm still willing to listen > to alternatives): It's well-established and known to a large part of > our target audience. It's pretty easy to determine from the version > number alone whether it's stable (minor version even) or experimental > odd); I wouldn't want to try to remember that--say--versions 2.2.1 > through 2.2.19 are experimental while 2.2.20 and up are stable. Just brainstorming: How about 2.3.a, 2.3.b, 2.3.c, ... as the experimental releases; 2.3.0 would then be the first stable release. Note that I agree that overall the problem lies more in getting people to try the experimental releases. See my "Developer Resources" post for more. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From pobrien@orbtech.com Mon Apr 8 23:51:49 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Mon, 8 Apr 2002 17:51:49 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081607.g38G7iD12531@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido van Rossum] > > > 1) Those people who want to be on the bleeding edge do not, by > > nature, care what release numbers mean. They'll work out of > > CVS repositories, or the similar "snapshot" systems, and can > > not be convinced otherwise. > > In Python's case, that's a vanishingly small fraction of the total > user base. Even the alpha/beta releases are downloaded by a > disappointingly small number of people. Many Python users apparently > consider themselves early adopters when they install 2.2 final when it > comes out. For every Python project that I'm involved with I work out of CVS. For Python itself, the lack of a C compiler for Windows keeps me from working with the CVS version. As the base of Windows users grows, I suspect this will become more common. I'd be more than happy to test the CVS version of Python if it was easier to do so on Win 98. --- Patrick K. O'Brien Orbtech From tim.one@comcast.net Mon Apr 8 23:52:44 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 08 Apr 2002 18:52:44 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15537.42265.890383.842760@12-248-41-177.client.attbi.com> Message-ID: [Skip Montanaro] > ... > Tim, can Windows installers be created by other people? By anyone with MSVC 6 and Wise 8.something. Few people have the latter. They need also to be careful that the specific versions of the MS components they ship (like msvcrt.dll) are officially blessed by MS for redistribution (e.g., the system's msvcrt.dll on my Win98SE box is, but the one I picked up on my Win2K box via a Win2K-specific service pack is not). I called for volunteers on c.l.py more than once to complete the start at an InnoSetup installer I checked in (InnoSetup is free-as-in-beer), but got no responses. From jeremy@zope.com Mon Apr 8 23:50:40 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 8 Apr 2002 18:50:40 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204082240.g38Me1n15697@pcp742651pcs.reston01.va.comcast.net> References: <2m7kni5rvk.fsf@starship.python.net> <200204081225.g38CPNn10186@mbuna.arbhome.com.au> <15537.56727.412033.844839@walden.zope.com> <200204082240.g38Me1n15697@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15538.7872.751836.382360@slothrop.zope.com> >>>>> "GvR" == Guido van Rossum writes: >> Basically, we need unambigious rules about when we'll stop doing >> micro releases from a particular branch. I think we should >> maintaince on >> 2.1 and 2.2, largely because 2.2 has so much experimental stuff. >> Once we get to 2.3, we can re-evaluate the situation and decide >> whether we want to continue maintaining 2.1. GvR> I think 2.1 is at the point where we only need to be reactive GvR> (e.g. fix reported core dumps). The focus should be to keep GvR> 2.2 alive -- sooner or later it will gain a reputation of GvR> stability. :-) Let me put my Python/Zope user hat on for a moment. I help maintain a Zope cluster that runs Python 2.1. We're not going to have another Zope release that uses Python 2.2 any time soon. When we do, it will probably be Zope 3 and that will be hard for me to upgrade to. So I expect the most recent Python I'll be using for the next year will be Python 2.1. I'd really be grateful for bug fixes and maintenance on this version as a result. Put in a slightly more abstract way, Python 2.2 is a major release with lots of new features. I expect a lot of users will stick with the last minor release that isn't a major release in disguise :-). So we'd serve them well be doing more maintenance on 2.1. Also, I haven't yet found any response to /F's suggestion that the std library always code to an earlier version of Python. What do you think of that? Jeremy From barry@zope.com Mon Apr 8 23:55:00 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Apr 2002 18:55:00 -0400 Subject: [Python-Dev] Re: Stability and change References: <15538.4237.873211.294420@12-248-41-177.client.attbi.com> Message-ID: <15538.8132.380580.109917@anthem.wooz.org> >>>>> "CP" == Christopher Petrilli writes: CP> What Barry is trying to say, and I most certainly agree with CP> his most rockiness, is that there shouldn't be oral tradition, CP> it should be obvious, on first inspection. I couldn't have (and didn't!) said it better meself. :) unstable-ly y'rs, -Barry From barry@zope.com Mon Apr 8 23:56:05 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Apr 2002 18:56:05 -0400 Subject: [Python-Dev] Re: Stability and change References: <200204081607.g38G7iD12531@pcp742651pcs.reston01.va.comcast.net> <200204082219.g38MJj614704@mercur.uphs.upenn.edu> <20020408224747.GF562@panix.com> Message-ID: <15538.8197.784050.208175@anthem.wooz.org> >>>>> "A" == Aahz writes: A> How about 2.3.a, 2.3.b, 2.3.c, ... as the experimental A> releases; 2.3.0 would then be the first stable release. Hey, let's keep them numeric and use 2.3.-1, 2.3.-2, etc. <-0 wink> -Barry From petrilli@amber.org Mon Apr 8 23:58:05 2002 From: petrilli@amber.org (Christopher Petrilli) Date: Mon, 08 Apr 2002 18:58:05 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204082231.g38MVNO14731@mercur.uphs.upenn.edu> Message-ID: On 4/8/02 6:31 PM, "Paul Hughett" wrote: > >> We could jump the micro release number to 100. [to mark the experimental to >> stable transition] > > -0 Why invent a new convention when a good one already exists? How do the > development groups other than Linux make this distinction? In the FreeBSD world (where I also participate), there is no such thing as an "experimental" release, such as the linux world might have. All "RELEASES" are stable, by definition. That's part of why it moves slower. There was just an announcement of a "Developer Preview" of 5.0, which is still 6 months away most likely. It is however, kept separate from the standard releases. This is the warning that accompanied the announcement: ***************************** WARNING ******************************** This is a development snapshot, and may include serious software bugs. Do not install this on a machine where important data may be put at risk. In addition, a number of debugging options are turned on by default, so the poor performance of this snapshot should not set expectations for the final release of 5.0. ********************************************************************** I think this is pretty clear as to what is intended for this release. I err not on the side of stasis, but on the side of deliberate progress. I was attracted to Python because its design was more deliberate, and its development more controlled than Perl. As Guido has pointed out, we can not give definite dates, just as the FreeBSD group can not (and recently had to push out 1 YEAR the release of 5.0). However, we could set "expected" dates, and perhaps "goals" for the release. At that point, people might be able to chart the waters better. As long as we keep those milestones in mind and publicly available, I think we are doing the best any open source organization can to meet the requirements of such a diverse populace. Chris From skip@pobox.com Tue Apr 9 00:01:02 2002 From: skip@pobox.com (Skip Montanaro) Date: Mon, 8 Apr 2002 18:01:02 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: <15538.4237.873211.294420@12-248-41-177.client.attbi.com> Message-ID: <15538.8494.31517.706412@12-248-41-177.client.attbi.com> Chris> The double-branch option is easier, and I know that when the Chris> FreeBSD group releases 4.5-RELEASE, it's been tested and solid, Chris> and will be supported by bug fixes for some period of time. They Chris> have only just recently stopped issue security patches for the Chris> 2.x tree. Is *-RELEASE a third branch, or is it what falls out the end of the *-STABLE pipe? Chris> It's just an idea, but I'm really opposed to magic numbers in Chris> determining whether something is a stable or experimental Chris> release. It doesn't matter to me either. Most of the discussion up to this point has been using Linux as an example of dual-track development. Since it uses even/odd numbering and most people on python-dev are at least familiar with it, it's a logical first choice to argue around. Maybe you do both. How about if all stable snapshots are named "stable-X.Y.Z" where Y just happens to always be even and all development snapshots are named "devel-X.Y.Z" where Y just happens to always be odd? Skip From martin@v.loewis.de Mon Apr 8 23:18:18 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 09 Apr 2002 00:18:18 +0200 Subject: [Python-Dev] Pondering some changes to python.c... In-Reply-To: <20020408150523.P11182@tummy.com> References: <20020407042038.O16962@tummy.com> <3CB155D1.422AD8C3@lemburg.com> <20020408142523.T16962@tummy.com> <200204082051.g38Kpv314922@pcp742651pcs.reston01.va.comcast.net> <20020408150523.P11182@tummy.com> Message-ID: Sean Reifschneider writes: > On Linux and a few other systems, setting the ps string requires re-writing > the block of memory starting at argv[0]. If the data in envp is > allocated next to argv, you can also make use of that (so that you can > write strings longer than "python testscript" in length). That's very hacky, IMO, and something I would not like to see generally available. If you need that feature, embed Python. Or, write an extension module allowing to rewrite argc/argv. ps manages to find out what the arguments are, so it should be possible to find (and overwrite) them even without getting them as arguments to main(). Regards, Martin From tim.one@comcast.net Tue Apr 9 00:00:32 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 08 Apr 2002 19:00:32 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > Because any PEP I write appears to mostly raise controversy once > brought up on c.l.py. This one was partly my fault: when I announced your PEP Parade essay, I added Note that Guido wants to keep 2.3 free of new core language features. at the start of the last paragraph. I did so because that's what I believed I had heard the day before. The next thing the community saw was a PEP from you introducing a new core language feature for 2.3. Some of the early responses specifically mentioned "wtf?! didn't Tim just say Guido wants yadda yadda?". BTW, the reaction to the sentence quoted above was crushingly favorable when it was first posted. The idea that 2.3 was to be a "consolidation" release was very popular. it's-not-bool's-fault-but-the-timing-was-perhaps-sub-optimal-ly y'rs - tim From jafo@tummy.com Tue Apr 9 00:06:57 2002 From: jafo@tummy.com (Sean Reifschneider) Date: Mon, 8 Apr 2002 17:06:57 -0600 Subject: [Python-Dev] Pondering some changes to python.c... In-Reply-To: ; from martin@v.loewis.de on Tue, Apr 09, 2002 at 12:18:18AM +0200 References: <20020407042038.O16962@tummy.com> <3CB155D1.422AD8C3@lemburg.com> <20020408142523.T16962@tummy.com> <200204082051.g38Kpv314922@pcp742651pcs.reston01.va.comcast.net> <20020408150523.P11182@tummy.com> Message-ID: <20020408170657.W11182@tummy.com> On Tue, Apr 09, 2002 at 12:18:18AM +0200, Martin v. Loewis wrote: >That's very hacky, IMO, and something I would not like to see >generally available. If you need that feature, embed Python. Indeed. Setting it in python.c is relatively easy... That's what the patch I included did... >ps manages to find out what the arguments are, so it should be possible >to find (and overwrite) them even without getting them as arguments to >main(). "ps" parses /proc//cmdline to figure out what the command-line was. AFAIK, /proc//cmdline is read-only: [root@prospero tmp]# date >/proc/$$/cmdline date: write error: Invalid argument I'm sure it's not hard to find the memory block that argv resides at if you are a kernel module, but I suspect fewer changes are required by using the mechanism I proposed earlier. ;-) Sean -- You know you're in Canada when: The weather reports starts with "All temperatures are degrees below zero unless otherwise stated." Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From jafo@tummy.com Tue Apr 9 00:10:16 2002 From: jafo@tummy.com (Sean Reifschneider) Date: Mon, 8 Apr 2002 17:10:16 -0600 Subject: [Python-Dev] Pondering some changes to python.c... In-Reply-To: <200204082146.g38LkOF15266@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Mon, Apr 08, 2002 at 05:46:24PM -0400 References: <20020407042038.O16962@tummy.com> <3CB155D1.422AD8C3@lemburg.com> <20020408142523.T16962@tummy.com> <200204082051.g38Kpv314922@pcp742651pcs.reston01.va.comcast.net> <20020408150523.P11182@tummy.com> <200204082146.g38LkOF15266@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020408171016.X11182@tummy.com> On Mon, Apr 08, 2002 at 05:46:24PM -0400, Guido van Rossum wrote: >Sigh. The more I hear about this, the more I think, too ugly, this >doesn't belong in Python. It's easier if we make the following restrictions: You can only set the process title to something the length of the original command-line or shorter (should work fine for simply deleting the "python" at the beginning). After calling sys.setproctitle(), C code which has ahold of argv will no longer have the original argv data. We should be able to have a copy of the original argv for regular python code. Are these reasonable restrictions to place on the setting of the process title? Sean -- One person's data is another person's program. -- Guy L. Steele, Jr. Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From petrilli@amber.org Tue Apr 9 00:10:22 2002 From: petrilli@amber.org (Christopher Petrilli) Date: Mon, 08 Apr 2002 19:10:22 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15538.8494.31517.706412@12-248-41-177.client.attbi.com> Message-ID: On 4/8/02 7:01 PM, "Skip Montanaro" wrote: > > Chris> The double-branch option is easier, and I know that when the > Chris> FreeBSD group releases 4.5-RELEASE, it's been tested and solid, > Chris> and will be supported by bug fixes for some period of time. They > Chris> have only just recently stopped issue security patches for the > Chris> 2.x tree. > > Is *-RELEASE a third branch, or is it what falls out the end of the *-STABLE > pipe? RELEASE is actually just a tag on the RELENG branch of the tree. For example, FreeBSD has the following branches in CVS for 4.x: RELENG_4 RELENG_4_3 RELENG_4_4 RELENG_4_5 On those, there are tags for the releases: RELENG_4_0_0_RELEASE RELENG_4_1_0_RELEASE RELENG_4_2_0_RELEASE RELENG_4_3_0_RELEASE And so on. This does create a more complex CVS environment than Python uses (and probably similar to what we used at Zope). This is not a bad thing, as it allows you to control things more carefully, but requires more CVS knowledge on the part of people involved. The following section of the handbook explains in some more detail about how the two branches progress through their life cycle: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/current-stable.htm l BTW, as to libraries and /F's comments... I think the library modules should always have to lag at least one release behind the interpreter. Maybe 2? This gives a couple of things: - No introducing weird problems in the library to compound compatibility issues - Iron out new features/changes by letting those people actually using them benefit/feel the pain, before we make the general populace deal. Again, I think stating what "policy" is about support for older releases is solves a HUGE amount of the problem. And probably a bigger percentage than a numbering scheme ever could. Chris From hughett@mercur.uphs.upenn.edu Tue Apr 9 00:10:30 2002 From: hughett@mercur.uphs.upenn.edu (Paul Hughett) Date: Mon, 8 Apr 2002 19:10:30 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15538.194.350128.498326@12-248-41-177.client.attbi.com> (message from Skip Montanaro on Mon, 8 Apr 2002 15:42:42 -0500) References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15538.194.350128.498326@12-248-41-177.client.attbi.com> Message-ID: <200204082310.g38NAUD14877@mercur.uphs.upenn.edu> Skip wrote: > I think an important question is what happens to the 2.2.x branch once > 2.4.0 is released? Should it die (in the sense of *never* getting > another micro release)? I think that would be a fair approach, > otherwise you have an ever-increasing support burden, trying to handle > more and more releases. Was there ever a huge clamor for 1.5.3? It > seems that for many people the heavens opened and Gabriel descended > with a 1.5.2 CD. ;-) I'd agree with that in the abstract. But then I realize that I'm still writing for 1.5.2, estimate that it would probably take a month or two to update 10K lines of C extension code to 2.0, 2.1, and 2.2, realize that I just don't have the time, and cringe away from the whole idea. My wish as an extension developer would be for Python to be stable for a year at a time, and then change to a new stable version that would last for another year. Then I could keep my extension code reasonably up to date by devoting one concentrated month per year. Maybe the problem is not as bad as I percieve it; but I can't afford the time to find out. That's why I like the idea of separate stable and experimental tracks--the language evolution happens and I get the benefits, but I don't have to constantly keep up with it. Paul Hughett From tim.one@comcast.net Tue Apr 9 00:45:18 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 08 Apr 2002 19:45:18 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204081825.g38IPpp13582@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > ... > I've started to call it the Logajan paradox: he wants his cake (zero > backwards incompatibilities) and eat it too (releases with new > features). It can't be done. I don't believe I ever heard James ask for a new feature, although he has allowed that new features might be OK provided that the Python version introducing them continues to run all older Python scripts without change. Rubin's view is more complicated, and I'm not sure it's been coherently expressed. Note that Logajan's view is hard to distinguish from what most ANSI/ISO language standard committees accept as their burden most of the time: no breakage of programs that conformed under the last standard, period. That's why, e.g., the C99 bool type was added via a pile of macro tricks, and *requires* #include'ing a new header file in any program that wants to use it as intended (the *actual* name of the new type is "_Bool", which is of a syntactic form C89 reserved to the language implementation; indeed, all the symbols Python uses internally of the form _Pxyz are forbidden to us by the std (Python's implementation is non-conforming all over the place); C99 couldn't add "bool" as a new keyword, but they could add _Bool, and then "#define bool _Bool" in a new standard header file; that can't break any existing code, because no existing code could possibly have #include'd the previously non-existent ; note too that older header files (like ) are forbidden from #include'ing stdbool.h on their own -- or, if they do, they have to hide all evidence of that from users). The ANSI/ISO committees pour person-decades into dreaming up ways to break nothing. It's not a crazy view on its own, but it's extremely expensive to do things that way (I've been on one ANSI committee, and worked closely with two others in previous lives -- it's excruciatingly slow going, and costs participants millions of dollars before it's over). It also requires extreme care in defining what "a conforming program" is, exactly. From greg@cosc.canterbury.ac.nz Tue Apr 9 00:51:23 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 09 Apr 2002 11:51:23 +1200 (NZST) Subject: [Python-Dev] ANN: Pyrex - a language for writing Python extension modules In-Reply-To: <20020404000900.L16962@tummy.com> Message-ID: <200204082351.LAA00573@s454.cosc.canterbury.ac.nz> Sean Reifschneider : > Neat! Looks like a pretty slick way of making C modules... Thanks for the encouragement! > My system preferred "#!/usr/bin/env python" instead of "#!/bin/env > python" in pyrexc. That seems to be acceptable on mine, too, so I'll change it. > A link to where you can find Plex would be nice on the main Pyrex > page. Excellent suggestion -- done now. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim.one@comcast.net Tue Apr 9 00:51:20 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 08 Apr 2002 19:51:20 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <02040820372900.14931@arthur> Message-ID: [Alex Martelli] > ... > Judging by languages such as C or C++, stability seems paramount; > yet Java churned a lot and still managed to spread a lot too (not > without a lot of help from high-$$$ marketing efforts, though). Not to be cynical , but what those all have in common is the visible backing of multi-billion dollar companies. Ditto Visual Basic from MS. Python will enjoy that status too just as soon as people start posting in outraged droves about Zope Corp's monopoly over the half-trillion dollar web content market. From guido@python.org Tue Apr 9 01:19:03 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 20:19:03 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 19:10:30 EDT." <200204082310.g38NAUD14877@mercur.uphs.upenn.edu> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15538.194.350128.498326@12-248-41-177.client.attbi.com> <200204082310.g38NAUD14877@mercur.uphs.upenn.edu> Message-ID: <200204090019.g390J3C15946@pcp742651pcs.reston01.va.comcast.net> > I'd agree with that in the abstract. But then I realize that I'm > still writing for 1.5.2, estimate that it would probably take a month > or two to update 10K lines of C extension code to 2.0, 2.1, and 2.2, > realize that I just don't have the time, and cringe away from the > whole idea. How did you arrive at this estimate? I'd be *very* surprised that a "mere" 10K lines of extension code would require significant work to keep it working under 2.2. I don't know where you get this idea, but Python's C API is actually quite stable, and more often than not, no changes are necessary to keep 1.5.2 code conformant with 2.2, or the changes are limited to 1 or 2 lines per file. > My wish as an extension developer would be for Python to be stable for > a year at a time, and then change to a new stable version that would > last for another year. Then I could keep my extension code reasonably > up to date by devoting one concentrated month per year. Maybe the > problem is not as bad as I percieve it; but I can't afford the time to > find out. I encourage you to give it a try. Just compile your code with the Python 2.2 header files, and see what kind of compilation errors you get. If you don't get compilation errors, most likely there's nothing you need to change. > That's why I like the idea of separate stable and experimental > tracks--the language evolution happens and I get the benefits, but I > don't have to constantly keep up with it. You really ought to try it for yourself. Whoever told you that it takes two months to update 10K lines of C extension code probably had never tried it -- or confused a *functional* upgrade of the code with merely keeping it working with a new Python release. (I almost feel offended, because I've been fighting so hard for backwards compatibility at the C API level.) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 9 01:26:46 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 20:26:46 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 19:10:22 EDT." References: Message-ID: <200204090026.g390Qku15961@pcp742651pcs.reston01.va.comcast.net> > BTW, as to libraries and /F's comments... I think the library > modules should always have to lag at least one release behind the > interpreter. Maybe 2? This gives a couple of things: > > - No introducing weird problems in the library to compound > compatibility issues Why would using a new feature introduce a weird problem in the library? Name one feature where this has happened in the past. > - Iron out new features/changes by letting those people actually > using them benefit/feel the pain, before we make the general > populace deal. I don't understand how not using a feature in the library means that the people actually using the feature feel the pain. To the contrary, starting to use a new feature in the library helps me understand the strengths and weaknesses of the feature. (I would make an exception for library modules that have a life outside the standard library, like /F's SRE. But there aren't many of those -- maybe the email package, although it works the other way around: it evolves outside the library until it's ready for inclusion.) > Again, I think stating what "policy" is about support for older > releases is solves a HUGE amount of the problem. And probably a > bigger percentage than a numbering scheme ever could. I'd be happy to state that we'll keep 2.1.x up to date whenever issues arrive, within the limitations of our resources. That probably means that we'll need a new community member to help backporting bugfixes. For 2.2.x I'm willing to make a stronger promise: I'd like to try to fix all bugs in 2.2.x as well as in 2.3, as long as 2.2 is considered more stable than 2.3 (this is trivially true until at least 2.3.1 is released :-). Resources are still a problem, so help is appreciated. But I think making 2.2.x as strong as possible sounds like a good goal. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Tue Apr 9 01:24:44 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 08 Apr 2002 20:24:44 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204082236.g38Maeg15671@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Barry] > Yup. Most people don't even know what kernel (or what gcc, etc.) they > have Indeed, I build a new Python almost daily, yet have no idea which Linux kernel is running under Win98SE (but I expect it's one of the "second edition"-- or was that "special edition?" --kernels). > -- all they know is that they installed Red Hat 7.2. I may have known that once -- my memory isn't what it used to ... something. From guido@python.org Tue Apr 9 01:32:45 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 20:32:45 -0400 Subject: [Python-Dev] Pondering some changes to python.c... In-Reply-To: Your message of "Mon, 08 Apr 2002 17:10:16 MDT." <20020408171016.X11182@tummy.com> References: <20020407042038.O16962@tummy.com> <3CB155D1.422AD8C3@lemburg.com> <20020408142523.T16962@tummy.com> <200204082051.g38Kpv314922@pcp742651pcs.reston01.va.comcast.net> <20020408150523.P11182@tummy.com> <200204082146.g38LkOF15266@pcp742651pcs.reston01.va.comcast.net> <20020408171016.X11182@tummy.com> Message-ID: <200204090032.g390Wjg15972@pcp742651pcs.reston01.va.comcast.net> > >Sigh. The more I hear about this, the more I think, too ugly, this > >doesn't belong in Python. > > It's easier if we make the following restrictions: > > You can only set the process title to something the length of the > original command-line or shorter (should work fine for simply deleting > the "python" at the beginning). This sounds awfully ill-defined. Do you know how this is accomplished on various Unix variants besides Linux? What if there's some platform where the rule is that the new length must be *exactly* the original length? Or where the limit is the original size minus 2 bytes? Does it work on Linux as an accident of the implementation, or is it guaranteed to work as Linux is upgraded? I'd hate to hear the complaints from users when they upgrade their kernel and find that it breaks Python, despite ABI compatibility promises. > After calling sys.setproctitle(), C code which has ahold of argv > will no longer have the original argv data. We should be able to > have a copy of the original argv for regular python code. What code would have a hold of argv, except main() which just calls Py_Main()? > Are these reasonable restrictions to place on the setting of the > process title? I presume the 2nd restriction could be a problem for C apps embedding Python. Maybe this should only be allowed when Python is the main interpreter; I wouldn't want a Python script to be able to change my argv[0] anyway if I were a large app (say Mozilla :-) embedding Python. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 9 01:38:47 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 20:38:47 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 19:00:32 EDT." References: Message-ID: <200204090038.g390clp15998@pcp742651pcs.reston01.va.comcast.net> > This one was partly my fault: when I announced your PEP Parade essay, I > added > > Note that Guido wants to keep 2.3 free of new core language features. > > at the start of the last paragraph. I did so because that's what I > believed I had heard the day before. Sounds like you need to be careful with the channeling there... :-) > The next thing the community saw was a PEP from you introducing a > new core language feature for 2.3. Some of the early responses > specifically mentioned "wtf?! didn't Tim just say Guido wants yadda > yadda?". BTW, the reaction to the sentence quoted above was > crushingly favorable when it was first posted. The idea that 2.3 > was to be a "consolidation" release was very popular. Maybe my mistake was thinking that adding bool was a small change. Maybe you could have brought this up when PEP 285 was first circulated on python-dev? I guess you didn't foresee the outcyr either. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 9 01:47:34 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 20:47:34 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 18:58:05 EDT." References: Message-ID: <200204090047.g390ldc16027@pcp742651pcs.reston01.va.comcast.net> > In the FreeBSD world (where I also participate), there is no such thing as > an "experimental" release, such as the linux world might have. All > "RELEASES" are stable, by definition. That's part of why it moves slower. That's what I like to believe of Python's "final" releases, too. > There was just an announcement of a "Developer Preview" of 5.0, which is > still 6 months away most likely. It is however, kept separate from the > standard releases. This is the warning that accompanied the announcement: > > ***************************** WARNING ******************************** > > This is a development snapshot, and may include serious software > bugs. Do not install this on a machine where important data may be > put at risk. In addition, a number of debugging options are turned on > by default, so the poor performance of this snapshot should not set > expectations for the final release of 5.0. > > ********************************************************************** How is this different from an alpha or beta release? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 9 01:55:15 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 20:55:15 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 18:50:40 EDT." <15538.7872.751836.382360@slothrop.zope.com> References: <2m7kni5rvk.fsf@starship.python.net> <200204081225.g38CPNn10186@mbuna.arbhome.com.au> <15537.56727.412033.844839@walden.zope.com> <200204082240.g38Me1n15697@pcp742651pcs.reston01.va.comcast.net> <15538.7872.751836.382360@slothrop.zope.com> Message-ID: <200204090055.g390tG016051@pcp742651pcs.reston01.va.comcast.net> [Jeremy] > Let me put my Python/Zope user hat on for a moment. I help maintain a > Zope cluster that runs Python 2.1. We're not going to have another > Zope release that uses Python 2.2 any time soon. When we do, it will > probably be Zope 3 and that will be hard for me to upgrade to. So I > expect the most recent Python I'll be using for the next year will be > Python 2.1. I'd really be grateful for bug fixes and maintenance on > this version as a result. Fair enough. > Put in a slightly more abstract way, Python 2.2 is a major release > with lots of new features. I expect a lot of users will stick with > the last minor release that isn't a major release in disguise :-). So > we'd serve them well be doing more maintenance on 2.1. Agreed -- with the caveat that we have fixed resources. > Also, I haven't yet found any response to /F's suggestion that the std > library always code to an earlier version of Python. What do you > think of that? I don't like it. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Tue Apr 9 01:53:17 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 08 Apr 2002 20:53:17 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204090038.g390clp15998@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Tim >> I did so because that's what I believed I had heard the day >> before. [Guido] > Sounds like you need to be careful with the channeling there... :-) It wasn't channeling at the time -- it was very clear the day before, at a PLabs meeting, when I asked what our plans were for 2.3, and all the "new feature" PEPs (at the time) were explicitly ruled out, in favor of consolidation. That you then introduced a "new feature" PEP of your own, and within 9 hours of the PEP Parade post, was utterly unexpected. > ... > Maybe my mistake was thinking that adding bool was a small change. It *is* a small change; I think the problem, at least at the start of the brouhaha, was merely that it was *a* change. More substantial complaints surfaced later, so it would have been a tempest regardless. > Maybe you could have brought this up when PEP 285 was first circulated > on python-dev? I guess you didn't foresee the outcyr either. :-) I posted my PEP Parade announcement to Python-Dev: http://mail.python.org/pipermail/python-dev/2002-March/020735.html and tried to tweak you about it before 24 hours had passed: http://mail.python.org/pipermail/python-dev/2002-March/020805.html As I've said before, I was raised in Wisconsin. If someone doesn't get the message the first time, it's too damn cold to stay outside all night long screaming -- and they're probably drunk anyway . From petrilli@amber.org Tue Apr 9 01:57:39 2002 From: petrilli@amber.org (Christopher Petrilli) Date: Mon, 08 Apr 2002 20:57:39 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204090047.g390ldc16027@pcp742651pcs.reston01.va.comcast.net> Message-ID: On 4/8/02 8:47 PM, "Guido van Rossum" wrote: >> In the FreeBSD world (where I also participate), there is no such thing as >> an "experimental" release, such as the linux world might have. All >> "RELEASES" are stable, by definition. That's part of why it moves slower. > > That's what I like to believe of Python's "final" releases, too. This is something I'd like to keep going, and something the numbering "scheme" would undermine. I trust that if it's "released" it's stable, regardless of whether it's compatible with what I'm doing. >> There was just an announcement of a "Developer Preview" of 5.0, which is >> still 6 months away most likely. It is however, kept separate from the >> standard releases. This is the warning that accompanied the announcement: >> >> ***************************** WARNING ******************************** >> >> This is a development snapshot, and may include serious software >> bugs. Do not install this on a machine where important data may be >> put at risk. In addition, a number of debugging options are turned on >> by default, so the poor performance of this snapshot should not set >> expectations for the final release of 5.0. >> >> ********************************************************************** > > How is this different from an alpha or beta release? As long as I've been in the FreeBSD world, I've never seen an alpha/beta release. What you get are the occasional Developer Preview (I think there is usually one before each X release (in X.Y). Then there are release candidates, usually 2-3. Otherwise, people are assumed to work off the trees directly, and there are tools to keep your system in check very easily. I have a machine here (dual Ppro, a bit antiquated, but it serves it's purpose) which CVSups (an automated CVS system) an updated environment every day, then builds the entire thing (which can take hours). It's painless for me, and I get to test some things I'm working on against the bleeding edge with little or no effort. In the FreeBSD world, you don't shove experimental code that hasn't gone through "some testing" into the CURRENT tree. It may not build everywhere, but it's gone through some testing, and usually will not cause anyone serious pain. I guess in the end, I see several competing interests here: - People who want minimal to no backward incompatibilities, ever - People who want to know that a release will be "supported" for some defined period - People who want the bleeding edge to be more available I would think that 90% of this can be solved with simple communications of what should be expected. It's not unreasonable to say that 2.1 will be supported with BUG FIXED (not features, bug fixes) until 2.4 or 2.5 is released. If we're on a 6-month "minor release" schedule, then that's roughly a year of stability. That seems generous. Chris -- | Christopher Petrilli | petrilli@amber.org From guido@python.org Tue Apr 9 02:04:07 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 21:04:07 -0400 Subject: [Python-Dev] Re: PEP Politics (was Re: Re: Stability and change) In-Reply-To: Your message of "Mon, 08 Apr 2002 18:43:59 EDT." <20020408224359.GE562@panix.com> References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <15537.11319.576212.846144@anthem.wooz.org> <15537.15490.631372.551394@12-248-41-177.client.attbi.com> <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> Message-ID: <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> > You may be right, but you misunderstood what I was suggesting: I'm > simply talking about looking at the PEP from the perspective of Joe > Stupid Pythonista, figuring out where the screams are likely to come > from, and clarify the language or write pre-emptive rebuttals. And I'm saying that with the current user base, *anything* from me will cause outrage. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 9 02:20:22 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 21:20:22 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 15:24:04 PDT." <20020408152404.B24764@glacier.arctrix.com> References: <20020408180043.GA2726@crystal.mems-exchange.org> <20020408113014.A24764@glacier.arctrix.com> <200204081842.g38Ig4k13782@pcp742651pcs.reston01.va.comcast.net> <20020408152404.B24764@glacier.arctrix.com> Message-ID: <200204090120.g391KMA16177@pcp742651pcs.reston01.va.comcast.net> > http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/current-stable.html That's funny. As Jeremy remarked, that's exactly how we use the CVS head. Maybe it *is* all a matter of perception. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From niemeyer@conectiva.com Tue Apr 9 02:31:13 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Mon, 8 Apr 2002 22:31:13 -0300 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204090038.g390clp15998@pcp742651pcs.reston01.va.comcast.net> References: <200204090038.g390clp15998@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020408223113.A1887@ibook.distro.conectiva> Guido, > Maybe my mistake was thinking that adding bool was a small change. I think sometimes you're excessively careful when publishing PEPs. When I first read your post about PEP 285, and read you asking if this should be implemented at all, I knew this would happen. This makes people which have no idea what they're talking about feel like they can influence something. Have a look at the ACCEPTED thread, and compare it to the first thread. In the second thread, you were decided to implement it. I'm not saying you shouldn't care about anyone's opinion, I just think you should publish PEPs as accepted, even if for some good reason, you fallback your idea (that's what really happens, anyway). -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From akuchlin@mems-exchange.org Tue Apr 9 02:39:17 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Mon, 8 Apr 2002 21:39:17 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204082240.g38Me1n15697@pcp742651pcs.reston01.va.comcast.net> References: <2m7kni5rvk.fsf@starship.python.net> <200204081225.g38CPNn10186@mbuna.arbhome.com.au> <15537.56727.412033.844839@walden.zope.com> <200204082240.g38Me1n15697@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020409013917.GA3135@ute.mems-exchange.org> On Mon, Apr 08, 2002 at 06:40:01PM -0400, Guido van Rossum wrote: >Hm, I think 2.1 may be a lost cause, but we've done this pretty >consistently for 2.2, and I strongly encourage to keep doing it. If >Michael can no longer do it, I'd like to call for a new volunteer, or >(if all else fails) have someone at PythonLabs take over. Make it a BDFL edict, then: bugfixes to the CVS HEAD should also be applied to the release22-maint branch by the same committer, unless the fix introduces an incompatibility. It's much more scalable to make everyone do an extra checkin every few days rather than to have some poor person dig through miles of CVS logs; maybe then we can roll out a new maintenance release with a week or so of checking, instead of a month or two. Michael, would it be worth posting the scripts you wrote to automatically apply patches to the 2.2-maint branch? Or putting them in Tools, or in nondist/? --amk (www.amk.ca) You're just an aging hippie, Professor. -- Ace, to the Doctor, in "The Greatest Show in the Galaxy" From neal@metaslash.com Tue Apr 9 02:45:00 2002 From: neal@metaslash.com (Neal Norwitz) Date: Mon, 08 Apr 2002 21:45:00 -0400 Subject: [Python-Dev] Release Schedules (was Stability & change) References: Message-ID: <3CB2479C.F5A475A4@metaslash.com> There seem to be two groups: 1) Early adopters: release early, release often; more features the better 2) The masses: easy does it, don't break anything, don't release so often I believe we can satisfy both groups--not perfectly, but good enough. This is partly summary and partly suggestion for terminology and schedule. I propose doing: Release name/type Version Schedule ----------------- ------- -------- Major releases 2.4 1.5 - 3 years Bugfix releases 2.2.1 ~ 3 months or as needed Development releases 2.3a0 ~ 3 months Don't get hung up on the version numbers, it doesn't matter. If we agree on the concept, we can name it later. 'Major' releases (roughly corresponding to Linux kernel even releases) would occur every ~ 18-36 months. These releases would be full executable, doc, etc. This seems to be the crux of what many people want. They want a vibrant changing language. But they don't want to have to deal with that change. They want longer cycles. We are talking about users of the language, not hard-core developers. These releases would still go through the alpha, beta, gamma releases. The last development release in a cycle would become the first alpha. Bugfix or point releases (eg, 2.x.y) would be made as needed, ~ 3 months. These releases would still go through the alpha, beta, gamma releases. This release would be concurrent with the development release. Since 2.2 is out there, I would suggest putting the bug fix effort only into it. There doesn't seem to be enough people to share the load for keeping 2.1 in good shape. Development releases, which are source only, could be released approximately every 3 months. These are basically alpha releases. They roughly correspond to Linux kernel odd releases. This would be to satisfy those that want new features or to test compatibility and are willing to build from source. These should be able to be released with very little effort. The benefits of this scheme is that I think it will appeal to a large majority of people regardless of whether they are an early-adopters or not. The only real drawback is that it will take more time to keep up the previous released version. However, I think this can be minimized by having everyone try to backport their work if it's a bug fix and have several people handle the patches for bug fix releases. I would volunteer to help Michael (or whoever rolls 2.2.2), although I don't have enough time to be the One. I don't think the numbering/labeling scheme is important. Does it really matter if we call the next release 2.4, 3.0, 2.3-RELEASE or whatever. I personally think the Linux versioning is the most well known and a bit more normal for most people over BSD. Neal From greg@cosc.canterbury.ac.nz Tue Apr 9 02:45:54 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 09 Apr 2002 13:45:54 +1200 (NZST) Subject: [Python-Dev] ANN: Pyrex - a language for writing Python extension modules In-Reply-To: <3CAC8886.9050302@campuspipeline.com> Message-ID: <200204090145.NAA00599@s454.cosc.canterbury.ac.nz> Kevin Butler : > 1- Any chance of having "typecast" spelled as one of: > > char*(value) #consistent w/ Python > (char *)value #consistent w/ C > cdef char*(value) #cdef + Python > (cdef char *)value #cdef + C > cdef(char*, value) #umm. Maybe easier for parsing? Probably not. I don't like the idea of overloading "cdef" that way, and the others are too hard to parse. Besides, I think it's a feature that typecasts look ugly. :-) If it helps, I have some features planned that will make it unnecessary to call malloc explicitly in most cases, which should remove a large source of the need for typecasts. > I think parsing C (& especially C++!) headers is likely to bring in > more headaches than it is worth. How deeply do you want to get into > macros, typedefs, other extern declarations, etc.? In the case of plain C, most of the machinery needed to parse and understand anything likely to be found in a .h is already there. Macros could be a problem, though. And I don't even want to think about C++ yet. > cdef extern class Grail: > # all members implicity "cdef extern" > int age > float volume > > # To self or not to self? > # Let declarations be consistent w/ C++? > def __init__( int age, float volume ) > def float quaff( float amount ) That will probably be cdef extern struct Grail: int age float volume Grail(int age, float volume) float quaff(float amount) i.e. C++ style constructor declaration, no self, and no def (which is reserved for declaring Python functions). Also "cdef struct", because "cdef class" will mean something different (that's how extension types will be defined). > (Next, I'll hit you up for exceptions and default args, then > overloading, then templates & RTTI... ) Oh, no... please... no... :-) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim.one@comcast.net Tue Apr 9 02:55:51 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 08 Apr 2002 21:55:51 -0400 Subject: [Python-Dev] Re: PEP Politics (was Re: Re: Stability and change) In-Reply-To: <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > And I'm saying that with the current user base, *anything* from me > will cause outrage. Ya, but when it's from the same people every time, other people do notice that. That Laura is such a crank . From guido@python.org Tue Apr 9 03:29:37 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 22:29:37 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 20:57:39 EDT." References: Message-ID: <200204090229.g392Tba17091@pcp742651pcs.reston01.va.comcast.net> > As long as I've been in the FreeBSD world, I've never seen an alpha/beta > release. What you get are the occasional Developer Preview (I think there > is usually one before each X release (in X.Y). Then there are release > candidates, usually 2-3. Otherwise, people are assumed to work off the > trees directly, and there are tools to keep your system in check very > easily. It's just in the naming then, nothing special. > I have a machine here (dual Ppro, a bit antiquated, but it serves > it's purpose) which CVSups (an automated CVS system) an updated > environment every day, then builds the entire thing (which can take > hours). It's painless for me, and I get to test some things I'm > working on against the bleeding edge with little or no effort. > > In the FreeBSD world, you don't shove experimental code that hasn't > gone through "some testing" into the CURRENT tree. It may not build > everywhere, but it's gone through some testing, and usually will not > cause anyone serious pain. Same for Python. We flame developers who break the build, and as a result this almost never happens (unless the tests pass on *their* machine but not elsewhere). We review code in the SF patch manager (painful). > I guess in the end, I see several competing interests here: > > - People who want minimal to no backward incompatibilities, ever Just say no to upgrades. > - People who want to know that a release will be "supported" for > some defined period I'd be happy to promise that release 2.X will be supported until 2.(X+2) (final) is released. > - People who want the bleeding edge to be more available I believe there are nightly checkout tarballs available from somewhere. But most people go directly to CVS. Anyway, I don't think this group has any problems with the current system. :-) > I would think that 90% of this can be solved with simple communications of > what should be expected. It's not unreasonable to say that 2.1 will be > supported with BUG FIXED (not features, bug fixes) until 2.4 or 2.5 is > released. If we're on a 6-month "minor release" schedule, then that's > roughly a year of stability. That seems generous. I think your math is off (2.4 would be 20 months) but something like that, yeah. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 9 03:35:06 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 22:35:06 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Mon, 08 Apr 2002 22:31:13 -0300." <20020408223113.A1887@ibook.distro.conectiva> References: <200204090038.g390clp15998@pcp742651pcs.reston01.va.comcast.net> <20020408223113.A1887@ibook.distro.conectiva> Message-ID: <200204090235.g392Z6n17116@pcp742651pcs.reston01.va.comcast.net> [me] > > Maybe my mistake was thinking that adding bool was a small change. [Gustavo] > I think sometimes you're excessively careful when publishing PEPs. :-) > When I first read your post about PEP 285, and read you asking if > this should be implemented at all, I knew this would happen. This > makes people which have no idea what they're talking about feel like > they can influence something. Have a look at the ACCEPTED thread, > and compare it to the first thread. In the second thread, you were > decided to implement it. I'm not saying you shouldn't care about > anyone's opinion, I just think you should publish PEPs as accepted, > even if for some good reason, you fallback your idea (that's what > really happens, anyway). Good idea. Though given the history, I don't think the outrage will be any worse next time. Some people just like to rally against "dictators". :-( --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 9 03:55:40 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 22:55:40 -0400 Subject: [Python-Dev] RELEASED: Python 2.1.3 Message-ID: <200204090255.g392tex17317@pcp742651pcs.reston01.va.comcast.net> Today, April 8th 2002, we've released Python 2.1.3, the latest version of Python 2.1. This is not the latest and greatest release, but is for people who want or need to run Python 2.1. Anthony Baxter was the 2.1.3 releasemeister, for which I thank him profusely. Great job, Anthony! The new release is available here: http://www.python.org/2.1.3/ (Note that we no longer release files via SourceForge -- it's a bit of a hassle, and few people seem to download their files from there.) This release has a small number of critical bug fixes that have come up since 2.1.2 was released, in particular one bug that caused Zope to crash under certain obscure circumstances. What's New in Python 2.1.3? --------------------------- Core - Fixed two bugs reported as SF #535905: under certain conditions, deallocating a deeply nested structure could cause a segfault in the garbage collector, due to interaction with the "trashcan" code; access to the current frame during destruction of a local variable could access a pointer to freed memory. - SF #457466: popenx() argument mangling hangs python (win9x only). Under certain circumstances, using any of the popen calls on win9x would cause python to hang. Library - The xml.sax.expatreader.ExpatParser class will no longer create circular references by using itself as the locator that gets passed to the content handler implementation. [SF bug #535474] - A security hole ("double free") was found in zlib-1.1.3, a popular third party compression library used by some Python modules. The hole was quickly plugged in zlib-1.1.4, and the Windows build of Python now ships with zlib-1.1.4. Windows - The installer now installs Start menu shortcuts under (the local equivalent of) "All Users" when doing an Admin install. - If you do what it takes to compile SSL support for Windows, a bug in httplib.py prevented SSL from working correctly. That bug has been fixed, but note that SSL support on Windows is still not enabled in this release. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 9 04:14:06 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 08 Apr 2002 23:14:06 -0400 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: Your message of "Mon, 08 Apr 2002 22:23:20 +0200." <02040822232006.14931@arthur> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <02040821105703.14931@arthur> <200204082000.g38K0O914299@pcp742651pcs.reston01.va.comcast.net> <02040822232006.14931@arthur> Message-ID: <200204090314.g393E6E17355@pcp742651pcs.reston01.va.comcast.net> [me] > > And there's the crux -- that one is still EXPERIMENTAL while the book > > is being written. [Alex] > That depends on how often the experimental branch is fed back into a > new stable one. If every 6 months, well, sure. But I think it > would not need to be anywhere as frequent. Neil suggests 18-36 months. Maybe we should put off 2.3 until, oh, late 2003? And plan to shove a whole lot more new features into it? And maybe move some minor features into the 2.2.x branch? (I hardly dare suggest bools. :-) > > So which distinction should we make? In a previous msg you disliked > > STABLE vs. CURRENT. Would you prefer STABLE vs. EXPERIMENTAL or > > CURRENT vs. EXPERIMENTAL? > > I think both sound very good in the abstract (while stable vs current > does not). It seems to me that stable vs experimental is slightly > preferable because it appears to draw a neater distinction, and > some connection of 'current' to other-than-stable might otherwise > attach from (e.g.) the BSD terminology. But there might be some > other consideration that doesn't come to my mind right now that > makes current/experimental preferable. OK, STABLE vs. EXPERIMENTAL it is. > There may not be ONE way out -- but maybe TWO tracks would do, > instead. On the stable track, releases that could break previously > correct code would be highly infrequent; on the experimental ones, > releases would be frequent -- ideally frequent enough that people > wouldn't go to CVS unless they're interested in contributing to the > internals, but could feel they're "on top of the leading-edge > RELEASE" (aka the experimental-track release). This also sounds similar to what Neil proposes. So maybe we're closing in to consensus: significantly slower "major" releases (like 2.3), more effort in "bugfix" releases (like 2.2.1) and more of those. And guess what, Anthony Baxter has offered to be the 2.2.2 releasemeister. I also like Andrew's idea of making everybody commit their changes in both branches -- to scale the effort of keeping the maintenance branch up-to-date. --Guido van Rossum (home page: http://www.python.org/~guido/) From Lance Ellinghaus" <3CB155D1.422AD8C3@lemburg.com> <20020408142523.T16962@tummy.com> <200204082051.g38Kpv314922@pcp742651pcs.reston01.va.comcast.net> <20020408150523.P11182@tummy.com> <20020408170657.W11182@tummy.com> Message-ID: <000001c1df7f$316c3420$3201040a@angel1> > I'm sure it's not hard to find the memory block that argv resides at if you > are a kernel module, but I suspect fewer changes are required by using the > mechanism I proposed earlier. ;-) This is not something that python should be doing. If you really need this, do it as an extension module. This functionality is too OS specific to be made a general change to Python when it provides no real advantage. Lance Ellinghaus From Lance Ellinghaus" <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <15537.11319.576212.846144@anthem.wooz.org> <15537.15490.631372.551394@12-248-41-177.client.attbi.com> <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> Message-ID: <000001c1df84$3e287340$3201040a@angel1> > > You may be right, but you misunderstood what I was suggesting: I'm > > simply talking about looking at the PEP from the perspective of Joe > > Stupid Pythonista, figuring out where the screams are likely to come > > from, and clarify the language or write pre-emptive rebuttals. > > And I'm saying that with the current user base, *anything* from me > will cause outrage. > > --Guido van Rossum (home page: http://www.python.org/~guido/) Guido, I would not worry so much about the current user base and their "feelings". People that want to use Python will. You have always led the direction of Python, and your choices have always been good for Python. That is why you are the BDFL. With your guidance and vision for Python, you have kept from turning into another Perl with all the "special case" language "features." There have been many changes people have wanted, that would have damaged the usability of the language even though they think it would have improved it. Most of them are still using Python even though they complained. I would not worry if people are outraged. Let them be. Most of the people that are outraged do not know the history of Python, the previous decisions that have been made. They just want it to be "their language." Continue to guide the development of Python and all will end up well. Lance Ellinghaus From fdrake@acm.org Tue Apr 9 06:32:56 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 9 Apr 2002 01:32:56 -0400 Subject: [Python-Dev] Developer resources (was Re: Stability and change) In-Reply-To: <20020408223655.GD562@panix.com> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> <15538.1515.580133.785503@12-248-41-177.client.attbi.com> <15538.2623.323978.125527@slothrop.zope.com> <20020408223655.GD562@panix.com> Message-ID: <15538.32008.560965.639327@grendel.zope.com> Aahz writes: > * Create a python-dev-tutor list for people who want to learn how to > participate in Python development. Not sure that's really the right approach. I'd rather see more effort go to persistent resources (like www.python.org/dev/; thanks Andrew!). Questions can reasonably be sent to python-dev and can feed the expansion & update of that material. > * Make the current CVS build available on some semi-public machine like > Starship (and by "available", I mean compiled). I like this idea. That could be done for both the development and maintenance branches (whatever we call them), and can be supported by volunteers on the starship. The "buildmeister" would not need to be a core developer at all, though perhaps should check the python-dev and python-checkins archives before deciding the time is right for a build. (BTW, is the Starship membership closed now? If so, the starship home page should be updated.) It would also be good to come up with an automated build-&-test procedure, so we can run something tindbox-style to keep a handle on state of the tree. This would be really good if we can get volunteers to provide build services on machines of different platforms. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From python@rcn.com Tue Apr 9 06:32:51 2002 From: python@rcn.com (Raymond Hettinger) Date: Tue, 9 Apr 2002 01:32:51 -0400 Subject: [Python-Dev] Release Schedules (was Stability & change) References: Message-ID: <000c01c1df87$fee0aa00$23b53bd0@othello> From: Neal Norwitz > There seem to be two groups: > > 1) Early adopters: release early, release often; more features the better > 2) The masses: easy does it, don't break anything, don't release so often > > I believe we can satisfy both groups--not perfectly, but good enough. > > This is partly summary and partly suggestion for terminology > and schedule. I propose doing: > > Release name/type Version Schedule > ----------------- ------- -------- > Major releases 2.4 1.5 - 3 years > Bugfix releases 2.2.1 ~ 3 months or as needed > Development releases 2.3a0 ~ 3 months Before deciding on an overly conservative response, consider whether the real issues are behind us. List comprehensions and the like were perceived as large changes. In contrast, a series of 6 months releases may be be more universally desired for the key things on the table like optimization, logging, and a more robust library. The anti-change cult may have drawn the line at augmented assignment and are prepared to rage for an eternity at print >>, but, even they, have not chafed at the new email module or the addition of pydoc or unittest. Further, they will surely welcome a refactored parser, an exposed parser, unified ints/longs, and optimized variable access. IOW, I believe that major, useful changes can be made without enraging the anti-change crowd. Improving the product, fixing bugs, expanding the library, filling in missing features, optimizing, and instrumenting aren't the issue. Just don't mess with the syntax and people won't freak. Raymond Hettinger P.S. The one area I'm less certain about is Deprecation. Phasing out lambda, map, and filter would please many but may have an equally strong counter-reaction. It's hard to tell (sometimes I think I'm the only one who like the functional stuff). P.S.S. I think the intensity of reaction to PEP 285 has to do with it being central to future programming style. It will affect and appear in programs across the board. Essentially, this proposal will be as pervasive as a change to the grammar would be. From jafo-python-dev@tummy.com Tue Apr 9 06:36:29 2002 From: jafo-python-dev@tummy.com (Sean Reifschneider) Date: Mon, 8 Apr 2002 23:36:29 -0600 Subject: [Python-Dev] Pondering some changes to python.c... In-Reply-To: <200204090032.g390Wjg15972@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Mon, Apr 08, 2002 at 08:32:45PM -0400 References: <20020407042038.O16962@tummy.com> <3CB155D1.422AD8C3@lemburg.com> <20020408142523.T16962@tummy.com> <200204082051.g38Kpv314922@pcp742651pcs.reston01.va.comcast.net> <20020408150523.P11182@tummy.com> <200204082146.g38LkOF15266@pcp742651pcs.reston01.va.comcast.net> <20020408171016.X11182@tummy.com> <200204090032.g390Wjg15972@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020408233629.Z11182@tummy.com> On Mon, Apr 08, 2002 at 08:32:45PM -0400, Guido van Rossum wrote: >This sounds awfully ill-defined. Do you know how this is accomplished >on various Unix variants besides Linux? What if there's some platform I got this mechanism by examining sendamil (which uses the ps output to display various states). Sendmail includes 6 mechanisms for setting the process name: pstat() psstrings (a struct you can set argc/argv through) sysmips() Apparently a SCO-specific mechanism which directly accesses kernel memory Overwriting the original argv[0] Setting argv[0] to point to a new buffer I have no idea how "published" one should consider these interfaces... >guaranteed to work as Linux is upgraded? I'd hate to hear the >complaints from users when they upgrade their kernel and find that it >breaks Python, despite ABI compatibility promises. Indeed. It's small consolation that it would also break sendmail... >What code would have a hold of argv, except main() which just calls >Py_Main()? In the normal python interpreter, nothing. My concern was more with a system that embeddeds Python... Which is why I'd lean more towards an implementation which required an explicit initialization call to enable. That would signal Python that the embedder knew of and condoned the modification of the process string. I'm torn between making it a build-time option in the SRPM (defaulting to disabled) and just dropping it as "too contentious". Sadly, the least objectional way to make it happen is probably making it os.pstat() and wrapping the pstat() system call on the platforms that have it. "Sadly" because Linux doesn't seem to be one of those platforms... Sean -- If you don't have time to do it right, when will you ever find time to do it over? Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From tim.one@comcast.net Tue Apr 9 07:15:45 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 09 Apr 2002 02:15:45 -0400 Subject: [Python-Dev] Developer resources (was Re: Stability and change) In-Reply-To: <15538.32008.560965.639327@grendel.zope.com> Message-ID: [Fred L. Drake, Jr.] > Not sure that's really the right approach. I'd rather see more effort > go to persistent resources (like www.python.org/dev/; thanks > Andrew!). Questions can reasonably be sent to python-dev and can feed > the expansion & update of that material. If anyone can make time to write guides, here are some specific points from a newer developer after tackling a small Python internals project, extracted from one of the emails I'm unlikely ever to answer (there isn't "a crisis" here, so it goes to the bottom of the stack): """ I was surprised at how many skills I needed to acquire to get this done: + editting .tex help files + communicating via SourceForge + learning to use CVS + finding where to put the unittests + learning what a context diff was + the ins and outs of METH_O + the subtleties of decref + the performance costs to tuple formation and arg parsing """ The good/bad news is that those things come up soooooo often that within a few weeks they'll forget they were ever a mystery. The barriers to entry are many; then again, the kind of code developer Python needs is someone obsessed enough to view that as a contemptible challenge . From aleax@aleax.it Tue Apr 9 07:16:57 2002 From: aleax@aleax.it (Alex Martelli) Date: Tue, 9 Apr 2002 08:16:57 +0200 Subject: [Python-Dev] Release Schedules (was Stability & change) In-Reply-To: <000c01c1df87$fee0aa00$23b53bd0@othello> References: <000c01c1df87$fee0aa00$23b53bd0@othello> Message-ID: <0204090816570D.14931@arthur> On Tuesday 09 April 2002 07:32, Raymond Hettinger wrote: ... > Before deciding on an overly conservative response, consider whether > the real issues are behind us. The intensity of the reaction to PEP 285 doesn't suggest that, in terms of perception. Yes, I see your last paragraph explaining that, but I only agree with it partways. > The anti-change cult may have drawn the line at augmented > assignment and are prepared to rage for an eternity at print >>, I'm not a member of any anti-change cult -- aug.ass. is good, list comprehensions great, etc, etc -- but I'm quite prepared to "rage for an eternity at print>>". You seem to be lumping people together too much. Cannot one examine each change on its merits and like or love many but still detest print>> ...? > welcome a refactored parser, an exposed parser, unified ints/longs, > and optimized variable access. FWIW, I was originally slightly wary of int/long unification, but having tried it out to the extent in which it's in 2.2 I like it now. I don't remember why I had been left with that wariness from (I think) Scheme experiences -- maybe too-slow machines back then, or careless arithmetic on my part at the time. > bugs, expanding the library, filling in missing features, optimizing, > and instrumenting aren't the issue. Just don't mess with the syntax > and people won't freak. So how do you explain the intensity of the reaction to Booleans? No doubt one could class them as "filling in missing features" -- whether that's "filling a badly needed gap", in Thomson's words, I won't take up again:-) -- but the reaction might uncharitably be described as "people freaking". Messing with the semantics may thus provoke intense reaction too. I think the 6-monthly rate of "stable" releases does have to do with it. If Booleans had been introduced into a declaredly experimental track reaction would have been far more muted. If what was planned for the next stable-track baseline was a feature that had been in the experimental track for over 6 months (field-proven etc), AND the stable track had first enjoyed a life of a year ot more of "no breaking previously correct code" stability, WITH the nice extras you mention (optimizing, instrumenting, etc), again I think grumbles might have been less strident. Schedules do matter, as does the perception of a stable, good porting target that is certain not to be due for obsolescence in six months. > P.S. The one area I'm less certain about is Deprecation. Phasing > out lambda, map, and filter would please many but may have an > equally strong counter-reaction. It's hard to tell (sometimes I > think I'm the only one who like the functional stuff). Again, over-lumping. I like map and filter well enough (though one can express them with list comprehensions, they're sometimes slightly more concise and faster). Not lambda though... the real functional stuff, sure, but lambda's underpowered and clunky. > P.S.S. I think the intensity of reaction to PEP 285 has to do with > it being central to future programming style. It will affect and > appear in programs across the board. Essentially, this proposal > will be as pervasive as a change to the grammar would be. These is some truth to that. Yet if it had not been introduced right into a release where many were looking forward to there being no such language-level changes, breaking the welcome perception of "some stability at last!", things might have been different. I think dual tracks for experimental and stable releases will help manage such things, giving the (not unfounded, mind you) perception of language stability to those who seek it (with an "actively supported release" -- instrumentation, optimizing, etc), AND bleeding-edge fast progress to neophiles (who are also so important to the community). Alex From tim.one@comcast.net Tue Apr 9 07:21:14 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 09 Apr 2002 02:21:14 -0400 Subject: [Python-Dev] Re: PEP Politics (was Re: Re: Stability and change) In-Reply-To: <000001c1df84$3e287340$3201040a@angel1> Message-ID: [Lance Ellinghaus, to Guido] > ... > You have always led the direction of Python, and your choices have > always been good for Python. No they haven't. In particular, his decision to accept your goofy rotormodule.c has gotten the PSF listed as a terrorist front by the US Government . > ... > Continue to guide the development of Python and all will end up well. Well, now that I think about it, I'm forced to agree after all. persuaded-against-his-will-ly y'rs - tim From tim.one@comcast.net Tue Apr 9 07:42:36 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 09 Apr 2002 02:42:36 -0400 Subject: [Python-Dev] Release Schedules (was Stability & change) In-Reply-To: <000c01c1df87$fee0aa00$23b53bd0@othello> Message-ID: [Raymond Hettinger] > ... > The anti-change cult may have drawn the line at augmented > assignment and are prepared to rage for an eternity at print >>, > but, I agree with Alex that this is too lumped together; not everyone who dislikes a specific new feature dislikes all new features. Not even most, and not even close. > even they, have not chafed at the new email module or the addition of > pydoc or unittest. New library modules are indeed cool with most people. > Further, they will surely welcome a refactored parser, That's of interest to almost nobody; PLabs wants it mostly to make it *possible* to do more intense program analysis without the code driving us insane (the current parser+codegen code is already pushed beyond its original design). > an exposed parser, The audience for that is also small, unless it's an all-things-to-all-people kind of parser. > unified ints/longs, People are definitely going to howl about finishing this one. People who used to strip the trailing 'L' from str(long) via str(long)[:-1] found themselves throwing away the last digit after str(long) stopped producing a trailing 'L'. What did they do instead? Almost every workaround I've seen amounted to doing repr(long)[:-1] instead. When repr(long) stops producing a trailing 'L' too, they're going to feel thrice screwed. That one's a special bitch, because it's neither the repr() nor the slice in isolation that creates the problem, so it's hard to give a sharp warning. Code is also going to break when, e.g., "1 << 100" stops producing 0, but at least that one can be warned about in a focused way. > and optimized variable access. People do have a mild appreciation for speedups. Forget what you read about this on c.l.py, though: the true market for speedups is smaller than you'd guess for all the whining about this that goes on. Of course it doesn't help that all the speedup work that's gone on since 1.5.2 (and yes, there has been some, and some of it difficult) has been more than wiped out by the expense of new features. So far, CVS 2.3 is significantly faster than 2.2.1, but we've got lots of time to blow that too . > IOW, I believe that major, useful changes can be made without > enraging the anti-change crowd. Improving the product, fixing > bugs, expanding the library, filling in missing features, optimizing, > and instrumenting aren't the issue. Just don't mess with the syntax > and people won't freak. Stick around . > P.S. The one area I'm less certain about is Deprecation. Phasing > out lambda, map, and filter would please many but may have an > equally strong counter-reaction. It would enjoy a strong counter-reaction, but so far none of those are deprecated. I'm delighted to see you didn't put "reduce" in that list (almost always both more obscure and slower than the obvious loop). > It's hard to tell (sometimes I think I'm the only one who like the > functional stuff). You're not. Guido wants all of your kind to go away, though . From aleax@aleax.it Tue Apr 9 07:44:58 2002 From: aleax@aleax.it (Alex Martelli) Date: Tue, 9 Apr 2002 08:44:58 +0200 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: <200204090314.g393E6E17355@pcp742651pcs.reston01.va.comcast.net> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <02040822232006.14931@arthur> <200204090314.g393E6E17355@pcp742651pcs.reston01.va.comcast.net> Message-ID: <0204090844580F.14931@arthur> On Tuesday 09 April 2002 05:14, Guido van Rossum wrote: ... > > That depends on how often the experimental branch is fed back into a > > new stable one. If every 6 months, well, sure. But I think it > > would not need to be anywhere as frequent. > > Neil suggests 18-36 months. Maybe we should put off 2.3 until, oh, > late 2003? And plan to shove a whole lot more new features into it? I think 18-36 months is a very reasonable rate for releases on the stable branch. I also think there should be an experimental branch that lets the "whole lot more new features" see the light of day and get some USE -- this would let you, as a language designer, experiment a bit, try things out, etc; lets the neophiles (and there ARE many!) play with the new toys; and accompanied with a guideline that a new language change doesn't get into a new stable-branch baseline unless it's seen some commensurate use in the experimental (say 2-3 months for minor things, more for bigger things) it would help everybody feel better about those changes that do make it into the new stable-branch baseline. Selfishly speaking -- I love new toys as much as the next man; it's just that a quarter century of experience has taught me to be wary of said toys in code that I, or some other poor soul, may then have to maintain forevermore. I think this is a common thing in people of our generation: selected to some extent for neo-phily (we did choose computer science when it was, yes, established, but nowhere as pervasive at today), but passed through the refiner's fire of decades of industry fashions and silver bullets:-). I'd just love horsing around with the bleeding edge experimental branch, helping shake bugs out and what not, AND at the same time rely on relative longevity for stuff I push out to customers. Many would no doubt be happy with the stable branches, a few would only go for the bleeding-edge stuff (can't see the Twisted guys as stooping to "stable", for example!-), but some of us would like a foot in both camps. > And maybe move some minor features into the 2.2.x branch? (I hardly > dare suggest bools. :-) What Raymond called optimizing and instrumenting, sure. Or if there was a way to let super be called with only self as an argument and figure out what class's method it's being called from (not that I know of a good one, but, YOU are the design genius around here...:-). A class of minor language changes which I think would be OK, if tried out in the experimental branch first, to put in the stable one: some language construct that used to be syntactically invalid or raise an exception and becomes valid and meaningful. There is some risk if try/except was used expecting the error, but in many cases I think it's a very small risk. That is different from a "silent change" of language where constructs are valid both before and after the change but behave differently, as in "print 2>1". Or at least so it seems to me. And PEP 246, but I still owe you one more rephrasing of what exactly it _means_ first:-). > OK, STABLE vs. EXPERIMENTAL it is. yea! > > internals, but could feel they're "on top of the leading-edge > > RELEASE" (aka the experimental-track release). > > This also sounds similar to what Neil proposes. > > So maybe we're closing in to consensus: significantly slower "major" > releases (like 2.3), more effort in "bugfix" releases (like 2.2.1) and > more of those. And guess what, Anthony Baxter has offered to be the > 2.2.2 releasemeister. I think the "experimental track" is missing from this picture. The resulting arrangement will still be a significant improvement for most people, but I think some (including my inner child yearning for new toys to play with AND _your_ inner child yearning for creative freedom:-) might be still happier if the experimental side was also there. But I may be missing something (74 more msgs to catch up with...). > I also like Andrew's idea of making everybody commit their changes in > both branches -- to scale the effort of keeping the maintenance branch > up-to-date. This reference to "both branches" does suggest there's an experimental one somewhere, thus leading to what I see as the best overall arrangement (and yes, dual commits seem feasible in many cases -- except for commits that need to go to experimental only, I guess). I'll catch up now... Alex From tim.one@comcast.net Tue Apr 9 07:49:43 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 09 Apr 2002 02:49:43 -0400 Subject: [Python-Dev] Release Schedules (was Stability & change) In-Reply-To: <0204090816570D.14931@arthur> Message-ID: [Alex Martelli] > ... > FWIW, I was originally slightly wary of int/long unification, but > having tried it out to the extent in which it's in 2.2 I like it now. > I don't remember why I had been left with that wariness from (I > think) Scheme experiences -- maybe too-slow machines back > then, or careless arithmetic on my part at the time. I'll suggest that what's saved you from that in Python so far is that division still "truncates" (in one way or another). As soon as you mix a few divides and adds in a Scheme that supports rationals (not all do, but most popular ones do), the magnitude of the denominators tends toward infinity faster than you can plug in more RAM chips. Guido and I both had this experience using ABC (which used unbounded rationals by default), and it was especially easy to fall into time/space difficulties in ABC because even "floating point" literals (like 6.02e23 and 3.017e-100) were treated as exact rationals. Guido assures me that latter was his idea, BTW. I'm still trying to find a way to forgive him -- it's not nearly enough that he'll never repeat that mistake . From martin@v.loewis.de Tue Apr 9 07:51:33 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 09 Apr 2002 08:51:33 +0200 Subject: [Python-Dev] Pondering some changes to python.c... In-Reply-To: <20020408170657.W11182@tummy.com> References: <20020407042038.O16962@tummy.com> <3CB155D1.422AD8C3@lemburg.com> <20020408142523.T16962@tummy.com> <200204082051.g38Kpv314922@pcp742651pcs.reston01.va.comcast.net> <20020408150523.P11182@tummy.com> <20020408170657.W11182@tummy.com> Message-ID: Sean Reifschneider writes: > >That's very hacky, IMO, and something I would not like to see > >generally available. If you need that feature, embed Python. > > Indeed. Setting it in python.c is relatively easy... That's what the > patch I included did... If you need that patch, just use it. But I would not like to see it in Python. > I'm sure it's not hard to find the memory block that argv resides at if you > are a kernel module, but I suspect fewer changes are required by using the > mechanism I proposed earlier. ;-) Fewest changes to Python can be made if there are no changes at all. If people then still need the feature, they might use the extension module that finds the argv block. Still no need to change the way how main() works. Regards, Martin From aleax@aleax.it Tue Apr 9 07:52:34 2002 From: aleax@aleax.it (Alex Martelli) Date: Tue, 9 Apr 2002 08:52:34 +0200 Subject: [Python-Dev] Release Schedules (was Stability & change) In-Reply-To: References: Message-ID: <0204090852340H.14931@arthur> On Tuesday 09 April 2002 08:42, Tim Peters wrote: ... > > unified ints/longs, > > People are definitely going to howl about finishing this one. People who > used to strip the trailing 'L' from str(long) via str(long)[:-1] found > themselves throwing away the last digit after str(long) stopped producing > a trailing 'L'. What did they do instead? Almost every workaround I've Giving strip methods an optional arguments, the characters that are OK to throw away (stopping the stripping when a not-to-be-thrown- away character shows up -- defaulting to whitespace like now of course), would help here and also with other related issues such as emulating Perl's "chomp". I also think it's the kind of change that would be OK in a minor release in the stable branch. Alex From martin@v.loewis.de Tue Apr 9 07:58:57 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 09 Apr 2002 08:58:57 +0200 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: <200204090314.g393E6E17355@pcp742651pcs.reston01.va.comcast.net> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <02040821105703.14931@arthur> <200204082000.g38K0O914299@pcp742651pcs.reston01.va.comcast.net> <02040822232006.14931@arthur> <200204090314.g393E6E17355@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > I also like Andrew's idea of making everybody commit their changes in > both branches -- to scale the effort of keeping the maintenance branch > up-to-date. For bug fixes, I encourage that. The question will come up whether new features also need to be back-ported. Regards, Martin From aleax@aleax.it Tue Apr 9 08:10:13 2002 From: aleax@aleax.it (Alex Martelli) Date: Tue, 9 Apr 2002 09:10:13 +0200 Subject: [Python-Dev] Release Schedules (was Stability & change) In-Reply-To: <3CB2479C.F5A475A4@metaslash.com> References: <3CB2479C.F5A475A4@metaslash.com> Message-ID: <0204090910130J.14931@arthur> On Tuesday 09 April 2002 03:45, Neal Norwitz wrote: > There seem to be two groups: > > 1) Early adopters: release early, release often; more features the > better 2) The masses: easy does it, don't break anything, don't release > so often > > I believe we can satisfy both groups--not perfectly, but good enough. I think this identifies the issues very precisely and proposes a good solution strategy. > > This is partly summary and partly suggestion for terminology > and schedule. I propose doing: > > Release name/type Version Schedule > ----------------- ------- -------- > Major releases 2.4 1.5 - 3 years > Bugfix releases 2.2.1 ~ 3 months or as needed > Development releases 2.3a0 ~ 3 months > > Don't get hung up on the version numbers, it doesn't matter. > If we agree on the concept, we can name it later. I fully agree on your concepts. I would suggest some renaming, for clearer/more incisive communication, but you're right, let's not get hang up on that: the substance of your proposal is _excellent_. > 'Major' releases (roughly corresponding to Linux kernel even releases) > would occur every ~ 18-36 months. These releases would be full > executable, doc, etc. This seems to be the crux of what > many people want. They want a vibrant changing language. But > they don't want to have to deal with that change. They want > longer cycles. We are talking about users of the language, > not hard-core developers. These releases would still go > through the alpha, beta, gamma releases. The last development release > in a cycle would become the first alpha. Perfect. > Development releases, which are source only, could be released > approximately every 3 months. These are basically alpha releases. > They roughly correspond to Linux kernel odd releases. This would > be to satisfy those that want new features or to test compatibility > and are willing to build from source. These should be able to be > released with very little effort. Not sure about the source-only part -- maybe there's some hacker wannabe on some Windows box out there that can't afford to buy VC++6. But I could volunteer to remedy that by compiling and making the win32 binaries available (not an installer: I wouldn't know how to write that, so this would have to be arranged) -- so it's not a killer. > I don't think the numbering/labeling scheme is important. Does it > really matter if we call the next release 2.4, 3.0, 2.3-RELEASE > or whatever. I personally think the Linux versioning is the most > well known and a bit more normal for most people over BSD. OK. I opine there's a little bit of importance in such "naming", but not all that much -- the substance matters more, and I love the substance of your proposal. Alex From aleax@aleax.it Tue Apr 9 08:26:30 2002 From: aleax@aleax.it (Alex Martelli) Date: Tue, 9 Apr 2002 09:26:30 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: Message-ID: <0204090926300L.14931@arthur> On Tuesday 09 April 2002 01:51, Tim Peters wrote: > [Alex Martelli] > > > ... > > Judging by languages such as C or C++, stability seems paramount; > > yet Java churned a lot and still managed to spread a lot too (not > > without a lot of help from high-$$$ marketing efforts, though). > > Not to be cynical , but what those all have in common is the > visible backing of multi-billion dollar companies. Ditto Visual Basic Not an issue of being cynical -- it's just a fact that C enjoyed no "visible backing" from AT&T (or did you have some other corp in mind?) to help it gain mindshare, and neither did C++. If "backing from multi-billion dollar corps" mattered as much as you seem to think we'd all be using PL/? or maybe some APL. Or do you forget who DID back _those_, and how much Big Blue dominated the computer scene until not-all-that-many years ago? Alex From Anthony Baxter Tue Apr 9 08:27:18 2002 From: Anthony Baxter (Anthony Baxter) Date: Tue, 09 Apr 2002 17:27:18 +1000 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: Message from martin@v.loewis.de (Martin v. Loewis) of "09 Apr 2002 08:58:57 +0200." Message-ID: <200204090727.g397RIo03169@mbuna.arbhome.com.au> >>> Martin v. Loewis wrote > > I also like Andrew's idea of making everybody commit their changes in > > both branches -- to scale the effort of keeping the maintenance branch > > up-to-date. > > For bug fixes, I encourage that. The question will come up whether new > features also need to be back-ported. There's a couple of simple cases: . Obvious bugs that should be backported. Where possible, the original committer should apply this. . Obvious new features that introduce new behaviour. This is something that shouldn't be applied automatically, but _if_ the feature is something that should be considered for the stable release then that should be flagged in the comment, but not applied to the branch straight away. Anthony From martin@v.loewis.de Tue Apr 9 08:31:53 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 09 Apr 2002 09:31:53 +0200 Subject: [Python-Dev] Release Schedules (was Stability & change) In-Reply-To: <0204090852340H.14931@arthur> References: <0204090852340H.14931@arthur> Message-ID: Alex Martelli writes: > Giving strip methods an optional arguments, the characters that are > OK to throw away (stopping the stripping when a not-to-be-thrown- > away character shows up -- defaulting to whitespace like now of course), > would help here and also with other related issues such as emulating Perl's > "chomp". I also think it's the kind of change that would be OK in a minor > release in the stable branch. This is http://python.org/sf/444708. Regards, Martin From tim.one@comcast.net Tue Apr 9 08:41:08 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 09 Apr 2002 03:41:08 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <0204090926300L.14931@arthur> Message-ID: [Alex Martelli] > Not an issue of being cynical -- it's just a fact that C enjoyed no > "visible backing" from AT&T (or did you have some other corp in > mind?) to help it gain mindshare, and neither did C++. > > If "backing from multi-billion dollar corps" mattered as much as you > seem to think we'd all be using PL/? or maybe some APL. Or do > you forget who DID back _those_, and how much Big Blue dominated > the computer scene until not-all-that-many years ago? No, I don't think backing matters much at all to how a language *becomes* popular. C and C++ and Java have both popularity and the backing of billion-dollar companies *now*, and big money wants to play with big money. This becomes very relevant when considering how big shops make adoption decisions: a small money player has an uphill battle even to get noticed. Programming languages, databases, web content management systems -- I don't care, it's all the same. Big shops want their suppliers to have deep pockets. Little shops can be more flexible, but a lot of them just copy what the big shops do. The only reason C got into big shops is that American management is so incompetent it didn't notice what kinds of risk the hippies in the IT department were taking <0.9 wink>. give-the-psf-a-billion-dollars-and-we'll-put-it-to-the-test-ly y'rs - tim From aleax@aleax.it Tue Apr 9 08:52:43 2002 From: aleax@aleax.it (Alex Martelli) Date: Tue, 9 Apr 2002 09:52:43 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: Message-ID: <0204090952430N.14931@arthur> On Tuesday 09 April 2002 09:41, Tim Peters wrote: ... > No, I don't think backing matters much at all to how a language *becomes* > popular. Ah, OK, we agree on that. Though it helped Java and VB. > C and C++ and Java have both popularity and the backing of > billion-dollar companies *now*, and big money wants to play with big Once something does become that popular, money flows more (again I see Java as different -- big money from the start). > lot of them just copy what the big shops do. The only reason C got into > big shops is that American management is so incompetent it didn't notice > what kinds of risk the hippies in the IT department were taking <0.9 > wink>. This is possible. It's not how it got into IBM (I remember the early times of it at IBM Research), but other managements might have been less careful or competent. > give-the-psf-a-billion-dollars-and-we'll-put-it-to-the-test-ly y'rs - IBM invested about that much in Linux technologies _after_ Linux had started earning huge popularity -- surely helped that popularity grow (IBM claims they're already roughly even in return from that investment through mainframe sales and consulting services -- Palmisano headed that effort, and he's IBM's #1 now, having been recently promoted). I may be wrong, but I perceive a slowly mounting group of entrepreneurs trying to make a billion (or whatever:-) from Python -- and quite ready to feed some of that moolah back, once the moolah IS there. We'll see... Alex From mwh@python.net Tue Apr 9 09:30:21 2002 From: mwh@python.net (Michael Hudson) Date: 09 Apr 2002 09:30:21 +0100 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Andrew Kuchling's message of "Mon, 8 Apr 2002 21:39:17 -0400" References: <2m7kni5rvk.fsf@starship.python.net> <200204081225.g38CPNn10186@mbuna.arbhome.com.au> <15537.56727.412033.844839@walden.zope.com> <200204082240.g38Me1n15697@pcp742651pcs.reston01.va.comcast.net> <20020409013917.GA3135@ute.mems-exchange.org> Message-ID: <2mzo0di89u.fsf@starship.python.net> Andrew Kuchling writes: > On Mon, Apr 08, 2002 at 06:40:01PM -0400, Guido van Rossum wrote: > >Hm, I think 2.1 may be a lost cause, but we've done this pretty > >consistently for 2.2, and I strongly encourage to keep doing it. If > >Michael can no longer do it, I'd like to call for a new volunteer, or > >(if all else fails) have someone at PythonLabs take over. > > Make it a BDFL edict, then: bugfixes to the CVS HEAD should also be > applied to the release22-maint branch by the same committer, unless > the fix introduces an incompatibility. > > It's much more scalable to make everyone do an extra checkin every few > days rather than to have some poor person dig through miles of CVS > logs; maybe then we can roll out a new maintenance release with a week > or so of checking, instead of a month or two. It's not that hard to backport fixes. The trawling through logs and regression testing is the hard bit, plus the release mechanics. Though abandoning sf will help here. > Michael, would it be worth posting the scripts you wrote to > automatically apply patches to the 2.2-maint branch? Or putting them > in Tools, or in nondist/? Maybe. They're rather specific to my email setup (though the VMers can probably adapt them easily enough), and I want to clean them up a bit first. Next week sometime, probably. Cheers, M. -- > say-hi-to-the-flying-pink-elephants-for-me-ly y'rs, No way, the flying pink elephants are carrying MACHINE GUNS! Aiiee!! Time for a kinder, gentler hallucinogen... -- Barry Warsaw & Greg Ward, python-dev From mwh@python.net Tue Apr 9 09:30:59 2002 From: mwh@python.net (Michael Hudson) Date: 09 Apr 2002 09:30:59 +0100 Subject: [Python-Dev] 2.2.1 release mechanics In-Reply-To: "Fred L. Drake, Jr."'s message of "Mon, 8 Apr 2002 16:56:19 -0400" References: <2m7knspj43.fsf@starship.python.net> <15538.1011.400324.537358@grendel.zope.com> Message-ID: <2mwuvhi88s.fsf@starship.python.net> "Fred L. Drake, Jr." writes: > Michael Hudson writes: > > April 10 ~1200 GMT: release ends > > > > By now, F, T & J have done their bits, uploaded > > files to creosote and sf (or pointed me to where I > > can get them), etc. > > Are we still on target for this? Yes. When can you have the docs done? Sean wants them for RPMs. Cheers, M. -- That's why the smartest companies use Common Lisp, but lie about it so all their competitors think Lisp is slow and C++ is fast. (This rumor has, however, gotten a little out of hand. :) -- Erik Naggum, comp.lang.lisp From mwh@python.net Tue Apr 9 09:31:48 2002 From: mwh@python.net (Michael Hudson) Date: 09 Apr 2002 09:31:48 +0100 Subject: [Python-Dev] 2.2.1 release mechanics In-Reply-To: Guido van Rossum's message of "Mon, 08 Apr 2002 17:52:40 -0400" References: <200204082152.g38LqeS15350@pcp742651pcs.reston01.va.comcast.net> Message-ID: <2mu1qli87f.fsf@starship.python.net> Guido van Rossum writes: > > + I believe we're no longer uploading anything to SF, although I'm > > not sure Michael and Anthony both believe that. > > AFAIK they were cc'ed on the msgs where that was decided. I agreed with them even though I was asleep when they arrived! Cheers, M. -- I would hereby duly point you at the website for the current pedal powered submarine world underwater speed record, except I've lost the URL. -- Callas, cam.misc From mwh@python.net Tue Apr 9 10:41:31 2002 From: mwh@python.net (Michael Hudson) Date: Tue, 9 Apr 2002 10:41:31 +0100 (BST) Subject: [Python-Dev] 2.2.1c2 RPMs In-Reply-To: <20020408235227.U16962@tummy.com> Message-ID: I'm going to post this to python-dev too, so people see the time line at the end. On Mon, 8 Apr 2002, Sean Reifschneider wrote: > On Mon, Apr 08, 2002 at 02:46:53PM +0100, Michael Hudson wrote: > >> I've uploaded the RPMs for 2.2.1rc2. This is what the final 2.2.1 > >> beta will be based on, so if you have any final comments let me > >> know. > > > >Does this mean you have new spec files to check in? > > Nearly... There are a couple of things that need to be done. I do have a > couple of questions on doing so though, perhaps you can help me with > them... I hope the other email cleared most of these up. > I'm expecting to spend tomorrow from 8pm to 11pm Mountain time (GMT+6) > working on the spec file and building RPMs. I'd like to get the new spec > file checked into the base release (and would like them to be able to build > Python RPMS directly from the tar file if possible). > > So, there are two things that I need to do: check the spec file into CVS, > and use that to build the final RPMS. > > I'd think that I won't really be able to build the final RPMs until the > "golden Python" is released. Or is there some "really shiny yellow but not > quite golden python" that I can build RPMs against, Yes. > but I can still check > in the spec files for inclusion in the final release? > > Sorry for the "basic python release" questions -- this is the first one > that I've been involved in before the fact. Hey, don't sweat it -- this is only my third... For clarity, here's how the next few hours go. All times GMT. 12:00 I declare the branch to be good, cut a tarball for you and leave it in creosote.python.org:~mwh/Python-2.2.1.tar.gz ~16:00 Fred finishes the docs and leaves a pile of files scattered all over creosote. 2:00-5:00 You grab the above, build some RPMs, and leave them in creosote.python.org:~jafo/python-2.2.1-foo.rpm. Emailing me md5s probably gets you brownie points. I sleep. 11:00 I move all files into place on creosote, edit web pages, write an announcement, etc. I go nowhere near sourceforge (hurrah!). 12:00 the release: I send the announcement, run make install on creosote, etc. Fireworks, champagne, and so on. Does this sound reasonable? Cheers, M. From mwh@python.net Tue Apr 9 10:49:00 2002 From: mwh@python.net (Michael Hudson) Date: 09 Apr 2002 10:49:00 +0100 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Skip Montanaro's message of "Mon, 8 Apr 2002 16:50:05 -0500" References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> <15538.1515.580133.785503@12-248-41-177.client.attbi.com> <15538.1632.252404.964819@anthem.wooz.org> <15538.4237.873211.294420@12-248-41-177.client.attbi.com> Message-ID: <2mr8lpi4mr.fsf@starship.python.net> Skip Montanaro writes: > BAW> Okay, so now I tell have to tell people that it works with any > BAW> version between 2.4.3 and 2.6.9, not including any 2.5.x release. > > Over time people should become aware that odd-numbered minor releases are > always development releases and are not to be relied upon. In which case you might as well not release "odd-numbered minor releases" becuase 90+% of the people who would use them get Python from CVS anyway. I would guess. Cheers, M. -- It *is*, however, rather juvenile, in contravention of the Trades Descriptions Act, and will lead eventually to the Dragon of Unhappiness flying up your bottom. -- Peter Ellis, ucam.chat From mwh@python.net Tue Apr 9 10:55:40 2002 From: mwh@python.net (Michael Hudson) Date: 09 Apr 2002 10:55:40 +0100 Subject: [Python-Dev] Developer resources (was Re: Stability and change) In-Reply-To: Tim Peters's message of "Tue, 09 Apr 2002 02:15:45 -0400" References: Message-ID: <2mn0wdi4bn.fsf@starship.python.net> Tim Peters writes: > If anyone can make time to write guides, here are some specific points from > a newer developer after tackling a small Python internals project, extracted > from one of the emails I'm unlikely ever to answer (there isn't "a crisis" > here, so it goes to the bottom of the stack): > > """ > I was surprised at how many skills I needed to acquire to get this done: > > + editting .tex help files > + communicating via SourceForge > + learning to use CVS > + finding where to put the unittests > + learning what a context diff was > + the ins and outs of METH_O > + the subtleties of decref > + the performance costs to tuple formation and arg parsing > """ Answer to most of these questions should be under http://www.python.org/dev/ somewhere, I guess. http://www.python.org/dev/tools.html has a few answers. Some belong in the docs. > The good/bad news is that those things come up soooooo often that within a > few weeks they'll forget they were ever a mystery. The barriers to entry > are many; then again, the kind of code developer Python needs is someone > obsessed enough to view that as a contemptible challenge . These are both good points. Cheers, M. -- "declare"? my bogometer indicates that you're really programming in some other language and trying to force Common Lisp into your mindset. this won't work. -- Erik Naggum, comp.lang.lisp From mwh@python.net Tue Apr 9 11:26:35 2002 From: mwh@python.net (Michael Hudson) Date: 09 Apr 2002 11:26:35 +0100 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: Anthony Baxter's message of "Tue, 09 Apr 2002 17:27:18 +1000" References: <200204090727.g397RIo03169@mbuna.arbhome.com.au> Message-ID: <2mk7rhi2w4.fsf@starship.python.net> Anthony Baxter writes: > . Obvious new features that introduce new behaviour. This is something > that shouldn't be applied automatically, but _if_ the feature is something > that should be considered for the stable release then that should be > flagged in the comment, but not applied to the branch straight away. What about recent pymalloc changes? They should yield some performance enhancements, will make no difference to Python code, but probably require cooperation from extension authors... Cheers, M. -- I think perhaps we should have electoral collages and construct our representatives entirely of little bits of cloth and papier mache. -- Owen Dunn, ucam.chat, from his review of the year From jafo-python-dev@tummy.com Tue Apr 9 11:27:09 2002 From: jafo-python-dev@tummy.com (Sean Reifschneider) Date: Tue, 9 Apr 2002 04:27:09 -0600 Subject: [Python-Dev] 2.2.1c2 RPMs In-Reply-To: ; from mwh@python.net on Tue, Apr 09, 2002 at 10:41:31AM +0100 References: <20020408235227.U16962@tummy.com> Message-ID: <20020409042709.B11182@tummy.com> On Tue, Apr 09, 2002 at 10:41:31AM +0100, Michael Hudson wrote: >I hope the other email cleared most of these up. Yes, it did. >Does this sound reasonable? Sounds great... Except that since I've spent a few hours this morning getting the 2.2.1 spec file ready to go, I'll probably use most of my 3 hour window getting 2.1.3 RPMs built. ;-) Thanks, Sean -- Maybe I'm a closet claustrophobic. -- Yosomono on #python Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From stephen@xemacs.org Tue Apr 9 11:59:25 2002 From: stephen@xemacs.org (Stephen J. Turnbull) Date: 09 Apr 2002 19:59:25 +0900 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15537.53736.695434.259081@anthem.wooz.org> References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> <15537.43727.369744.356521@grendel.zope.com> <15537.47467.132247.414879@anthem.wooz.org> <200204081628.g38GSet12746@pcp742651pcs.reston01.va.comcast.net> <15537.53736.695434.259081@anthem.wooz.org> Message-ID: <87lmbxdto2.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "GvR" == Guido van Rossum writes: GvR> For most mortals, the XEmacs package manager simply *sucks*. GvR> Maybe that is because it's broken out of the box and you have GvR> to find the right webpage that tells you how to manually fix GvR> some things. But it still sucks. >>>>> "BAW" == Barry A Warsaw writes: BAW> I agree totally, but that's a quality of implementation BAW> problem. Once it works, it's (almost) great. I wish. There are real problems waiting to bite us, besides the gratuitous menu changes and implementation sillinesses, such as its dependence on EFS. The main one that is Python-planning-relevant is that you guys have standard libraries nested several deep all over the place, no? We have about five packages that other packages consider "a lot closer to core than I am": APEL (used by many), EFS (by the package manager itself), semantic/eieio (by JDE), and widget.el (used by just about everything via the Customize facility). JDE, the Java Development Environment, is one of our big problem children because it insists on being distributed with its own versions of semantic and eieio. Fortunately, nothing else much uses them. EFS (Extended File System) is well-done and well-maintained, but unfortunately by design depends on external FTP implementations, and every few months Red Hat or SuSE "does something" to FTP (kerberize, whatever) and EFS loses on that platform---as does our package manager. APEL is "A Portable Emacs Library", and what it does is smooth over backwards compatibility issues. Unfortunately, it does it in ways that (a) tend to introduce forward compatibility issues and (b) not really visible to debuggers. This leads to embarrassments where A Leading Developer says on Usenet that XEmacs supports such-and-so API from GNU Emacs 21, and then discovers that it ain't so, it's actually subtly different code from APEL---only the docstrings are the same. The widgets are pretty stable, but they're only loosely coupled with anything besides Custom. In other words, the main reason our package system has been so successful is that we simply don't have to worry about the issues this thread is about. But in a few cases where we do, we tend to lose. So be cautious. I won't say our experience proves it can't work; I'd actually rather say the glass is half-full, and I think that you guys can fill it better than we have so far. But it won't come "for free", there's more conceptual work to do than just "do it better than XEmacs." And Andrew Kuchling mentions Debian apt, well, that's the direction a number of the library developers are heading in terms of wishlists. For all that, it's been a roaring success with our users. Fortunately-for-them-the-BDFL-is-not-our-BDFL-ly y'rs, -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Don't ask how you can "do" free software business; ask what your business can "do for" free software. From jacob@strakt.com Tue Apr 9 12:35:58 2002 From: jacob@strakt.com (=?ISO-8859-1?Q?Jacob_Hall=E9n?=) Date: Tue, 09 Apr 2002 13:35:58 +0200 Subject: [Python-Dev] Multiple revisions [was: Re: Stability and change] Message-ID: <3CB2D21E.1000804@strakt.com> Guido van Rossum wrote: >>/ It would help to some degree if the "stable" releases could be shown />>/ to have SOME kind of enhancement. Not as much as if language />>/ changes which did not break existing correct code could also be />>/ there, but some. Modules such as email are a big part of the draw />>/ of 2.2, for example (not quite as big as language-level enhancements= , />>/ but big). /> > Now there's a good proposal. But it costs time and effort that takes > away from our main mission (and PythonLabs is already under a lot of > pressure). I wonder if there's a commercial market for this? Maybe > ActiveState could back-port important packages or language changes and > insert them into ActivePython? You can significantly reduce the time needed to apply patches to several versions of your software by using a smart revision control system. I recently discovered "arch", which has been designed to handle multiple parallell versions. It is a marvel, and it comes with really good descriptions of how to handle the problem in practice. I know that Linus Torvalds is considering it for the Linux kernel. http://regexps.com/ Jacob Hall=E9n From mwh@python.net Tue Apr 9 13:00:11 2002 From: mwh@python.net (Michael Hudson) Date: 09 Apr 2002 13:00:11 +0100 Subject: [Python-Dev] 2.2.1 is on the move. In-Reply-To: Michael Hudson's message of "Tue, 9 Apr 2002 10:41:31 +0100 (BST)" References: Message-ID: <2melhpw08k.fsf@starship.python.net> Michael Hudson writes: > For clarity, here's how the next few hours go. All times GMT. > > 12:00 I declare the branch to be good, cut a tarball for you and leave it > in creosote.python.org:~mwh/Python-2.2.1.tar.gz This is now. I've tagged (r221) the tree and put a tarball in my home dir on creosote. e7012d611602b62e36073c2fd02396a3 Python-2.2.1.tgz I reserve the right to move the tag later, but I really hope I don't have to. > ~16:00 Fred finishes the docs and leaves a pile of files scattered all > over creosote. You next :) Time to start poking around pydotorg... Cheers, M. -- Any form of evilness that can be detected without *too* much effort is worth it... I have no idea what kind of evil we're looking for here or how to detect is, so I can't answer yes or no. -- Guido Van Rossum, python-dev From martin@v.loewis.de Tue Apr 9 13:03:21 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 09 Apr 2002 14:03:21 +0200 Subject: [Python-Dev] Multiple revisions [was: Re: Stability and change] In-Reply-To: <3CB2D21E.1000804@strakt.com> References: <3CB2D21E.1000804@strakt.com> Message-ID: Jacob Hall=E9n writes: > You can significantly reduce the time needed to apply patches to several > versions of your software by using a smart revision control system. I don't believe that statement (unless CVS classifies as "smart" already). It takes me a minute or less to commit a patch in the 2.2 maintainance branch if no actual backporting is needed. If you need backporting, the overhead of the RCS becomes insignificant. Regards, Martin From jacob@strakt.com Tue Apr 9 13:44:52 2002 From: jacob@strakt.com (=?ISO-8859-1?Q?Jacob_Hall=E9n?=) Date: Tue, 09 Apr 2002 14:44:52 +0200 Subject: [Python-Dev] "Unstable" is an ambiguous word... Message-ID: <3CB2E244.6010102@strakt.com> Guido van Rossum wrote: > ... > OK, STABLE vs. EXPERIMENTAL it is. > >>/ There may not be ONE way out -- but maybe TWO tracks would do, />>/ instead. On the stable track, releases that could break previously />>/ correct code would be highly infrequent; on the experimental ones, />>/ releases would be frequent -- ideally frequent enough that people />>/ wouldn't go to CVS unless they're interested in contributing to the />>/ internals, but could feel they're "on top of the leading-edge />>/ RELEASE" (aka the experimental-track release). /> > This also sounds similar to what Neil proposes. > > So maybe we're closing in to consensus: significantly slower "major" > releases (like 2.3), more effort in "bugfix" releases (like 2.2.1) and > more of those. And guess what, Anthony Baxter has offered to be the > 2.2.2 releasemeister. I like this. I would also like to add that there is a benefit with the multiple number= ing scheme. The various distribution makers will most probably build packages for bot= h the stable and the experimental versions, allowing people to easily maintain both versions on their machines. With the current situation, most distrib= ution builders will skip the alpha, beta and rc versions, thus decreasing the population that will actually test them. Jacob Hall=E9n From aleax@aleax.it Tue Apr 9 13:59:15 2002 From: aleax@aleax.it (Alex Martelli) Date: Tue, 9 Apr 2002 14:59:15 +0200 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: <3CB2E244.6010102@strakt.com> References: <3CB2E244.6010102@strakt.com> Message-ID: On Tuesday 09 April 2002 02:44 pm, Jacob Hallén wrote: ... > I would also like to add that there is a benefit with the multiple > numbering scheme. The various distribution makers will most probably build > packages for both the stable and the experimental versions, allowing people > to easily maintain both versions on their machines. With the current > situation, most distribution builders will skip the alpha, beta and rc > versions, thus decreasing the population that will actually test them. I suspect the situation may change by distribution. Mandrake, if I know them well (and I'm a satisfied customer for over a year, supporting member of the Mandrake club, etc), WILL keep on top of the bleeding edge, releasing a stable-python and an experimental-python package, or something like that -- they know their customer base includes a lot of rabid neophiles, as well as people who only want stability. Debian, though, I wonder -- I doubt they'll put in their "Stable" distribution anything not vouchsafed for by Methushelah in person:-). As for RedHat, no problem -- they'll keep shipping 1.5.2 anyway...!-) Alex From fdrake@acm.org Tue Apr 9 14:30:02 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 9 Apr 2002 09:30:02 -0400 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <02040821105703.14931@arthur> <200204082000.g38K0O914299@pcp742651pcs.reston01.va.comcast.net> <02040822232006.14931@arthur> <200204090314.g393E6E17355@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15538.60634.808936.3925@grendel.zope.com> Martin v. Loewis writes: > For bug fixes, I encourage that. The question will come up whether new > features also need to be back-ported. I think it would be a good idea to have a way to determine from a checkin comment whether it was part of a bugfix (w/ issue # if there is one), feature (w/ issue #, again), or simple code cleanliness (whitespace cleanup, etc.). Something that can be checked by tools would be nice, especially for release managers. It might also be worth adding something that does the appropriate whitespace checks automatically and requires that the whitespace be right before allowing the commit, if only to avoid having the separate whitespace cleanups. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From hughett@mercur.uphs.upenn.edu Tue Apr 9 14:39:13 2002 From: hughett@mercur.uphs.upenn.edu (Paul Hughett) Date: Tue, 9 Apr 2002 09:39:13 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15538.6867.127517.168835@slothrop.zope.com> (message from Jeremy Hylton on Mon, 8 Apr 2002 18:33:55 -0400) References: <200204081607.g38G7iD12531@pcp742651pcs.reston01.va.comcast.net> <200204082219.g38MJj614704@mercur.uphs.upenn.edu> <15538.6867.127517.168835@slothrop.zope.com> Message-ID: <200204091339.g39DdDa16113@mercur.uphs.upenn.edu> Jeremy wrote: > Paul, Do you ever do build Python from a cvs checkout? No. I'm doing production work with Python, so it makes better sense for me to stick to the stable versions. I'm glad the experimental versions are there so the language develops, but I don't follow them myself. > It seems to me that what you call an experimental track, we call a > "cvs checkout." What you call a stable releases, we call a release. Could be, though lurking on python-dev gives me the impression of a lot more language change than fits my interpretation of "stable version". As a user of the stable track, I'm not particularly concerned whether the experimental versions come out as CVS updates or numbered versions. The language developers and bleeding-edge experimenters can work that one out among themselves. What I want (as I said in a message you probably haven't read yet) is a series of stable versions that I can use with few modifications for a year or so, interspersed with periods of intense change updating to the new stable version. Updating to a new stable version every six months is too fast for me. > I think all of this [being a stable version] was true of Python 2.1 > & 2.2 and will be true of 2.3. I think the problem here is that discovering this is difficult. I just looked at the Python home page. Between that and the downloads page, there are 7 distinct versions listed, but no clues to the newbie as to which one he ought to use. Perhaps we should add a short paragraph that says "If you want the leading edge, use the highest-numbered version there is. (And please report all the bugs you discover.) If you want a stable version for production use, use the latest in the 2.2 series." > Sure, there are bugs in 2.1 & 2.2, but we fixed the ones we knew how > to fix. I expect that "stable" Linux kernel releases also have bugs, > else I wouldn't have a 2.2.12 release on my home machine :-). Yes, they do, but generally I don't need to upgrade to the latest kernel unless it fixes a bug that hurts me or need the latest security fix. That's part of what "stable version" means to me. Paul Hughett From hughett@mercur.uphs.upenn.edu Tue Apr 9 14:46:27 2002 From: hughett@mercur.uphs.upenn.edu (Paul Hughett) Date: Tue, 9 Apr 2002 09:46:27 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <20020408224747.GF562@panix.com> (message from Aahz on Mon, 8 Apr 2002 18:47:47 -0400) References: <200204081607.g38G7iD12531@pcp742651pcs.reston01.va.comcast.net> <200204082219.g38MJj614704@mercur.uphs.upenn.edu> <20020408224747.GF562@panix.com> Message-ID: <200204091346.g39DkRZ16121@mercur.uphs.upenn.edu> aahz wrote: > How about 2.3.a, 2.3.b, 2.3.c, ... as the experimental releases; 2.3.0 > would then be the first stable release. That would give us a simple rule: Production users should strictly avoid any release number that isn't all numbers. Paul Hughett From fdrake@acm.org Tue Apr 9 14:56:41 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 9 Apr 2002 09:56:41 -0400 Subject: [Python-Dev] 2.2.1 release mechanics In-Reply-To: <2mwuvhi88s.fsf@starship.python.net> References: <2m7knspj43.fsf@starship.python.net> <15538.1011.400324.537358@grendel.zope.com> <2mwuvhi88s.fsf@starship.python.net> Message-ID: <15538.62233.177329.83289@grendel.zope.com> Michael Hudson writes: > Yes. When can you have the docs done? Sean wants them for RPMs. It won't take long, but I am looking at the patch to produce info files. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From skip@pobox.com Tue Apr 9 15:10:45 2002 From: skip@pobox.com (Skip Montanaro) Date: Tue, 9 Apr 2002 09:10:45 -0500 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: <200204090314.g393E6E17355@pcp742651pcs.reston01.va.comcast.net> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <02040821105703.14931@arthur> <200204082000.g38K0O914299@pcp742651pcs.reston01.va.comcast.net> <02040822232006.14931@arthur> <200204090314.g393E6E17355@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15538.63077.361526.769992@12-248-41-177.client.attbi.com> Guido> So maybe we're closing in to consensus: significantly slower Guido> "major" releases (like 2.3), more effort in "bugfix" releases Guido> (like 2.2.1) and more of those. And guess what, Anthony Baxter Guido> has offered to be the 2.2.2 releasemeister. Silly boy... :-) Are bugfix releases going to require less effort to produce? That is, will they just be a source tarball? Guido> I also like Andrew's idea of making everybody commit their Guido> changes in both branches -- to scale the effort of keeping the Guido> maintenance branch up-to-date. That makes sense, though for those of us who've never tried checking in on multiple branches someone might want to give a quick tutorial. Is it as simple has having two trees checked out at once, one checked out using -r and one using -A? Skip From hughett@mercur.uphs.upenn.edu Tue Apr 9 15:20:25 2002 From: hughett@mercur.uphs.upenn.edu (Paul Hughett) Date: Tue, 9 Apr 2002 10:20:25 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204090019.g390J3C15946@pcp742651pcs.reston01.va.comcast.net> (message from Guido van Rossum on Mon, 08 Apr 2002 20:19:03 -0400) References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15538.194.350128.498326@12-248-41-177.client.attbi.com> <200204082310.g38NAUD14877@mercur.uphs.upenn.edu> <200204090019.g390J3C15946@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204091420.g39EKPF16144@mercur.uphs.upenn.edu> Guido wrote: > How did you arrive at this estimate? [two months to upgrade a C > extension to 2.2] Actually, "guess" would be a better word than "estimate". I've never done this before, have scant information about what changes might be required, and have learned from bitter experience that it's better to be pessimistic in such circumstances. > I'd be *very* surprised that a "mere" 10K lines of extension code > would require significant work to keep it working under 2.2. I don't > know where you get this idea, but Python's C API is actually quite > stable, and more often than not, no changes are necessary to keep > 1.5.2 code conformant with 2.2, or the changes are limited to 1 or 2 > lines per file. That's good news, and moves that task noticeably higher in my to-do list. > (I almost feel offended, because I've been fighting so hard for > backwards compatibility at the C API level.) No offense meant, and I thank you for that effort. The problem may be more that that is not visible as a policy, except to insiders. Now I have to confess that, while I've downloaded the 2.0 source package, I've been too busy actually using Python to open it up and look at the API documentation. When I do that, there are two things that I would be really happy to see: Some prose like you just wrote to me, explaining that you are really serious about compatibility, and that version-to-version upgrades are generally pretty easy. And a detailed list of the API changes made, so that I can review it and say, "No problem there. Not there either. Oops, better fix that..." and be able to get a reasonable estimate of the effort required for an upgrade. If you've already done this, I apologize; as I said, I haven't yet spent much time working on (or even looking at) upgrading to 2.x yet. Paul Hughett From guido@python.org Tue Apr 9 15:18:56 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 10:18:56 -0400 Subject: [Python-Dev] Pondering some changes to python.c... In-Reply-To: Your message of "Mon, 08 Apr 2002 23:36:29 MDT." <20020408233629.Z11182@tummy.com> References: <20020407042038.O16962@tummy.com> <3CB155D1.422AD8C3@lemburg.com> <20020408142523.T16962@tummy.com> <200204082051.g38Kpv314922@pcp742651pcs.reston01.va.comcast.net> <20020408150523.P11182@tummy.com> <200204082146.g38LkOF15266@pcp742651pcs.reston01.va.comcast.net> <20020408171016.X11182@tummy.com> <200204090032.g390Wjg15972@pcp742651pcs.reston01.va.comcast.net> <20020408233629.Z11182@tummy.com> Message-ID: <200204091418.g39EIup20292@odiug.zope.com> [me] > >This sounds awfully ill-defined. Do you know how this is accomplished > >on various Unix variants besides Linux? What if there's some platform [Sean] > I got this mechanism by examining sendamil (which uses the ps output to > display various states). Sendmail includes 6 mechanisms for setting the > process name: > > pstat() > psstrings (a struct you can set argc/argv through) > sysmips() > Apparently a SCO-specific mechanism which directly accesses kernel memory > Overwriting the original argv[0] > Setting argv[0] to point to a new buffer > > I have no idea how "published" one should consider these interfaces... Oh, yuck. Bah. Please go away. :-) > >guaranteed to work as Linux is upgraded? I'd hate to hear the > >complaints from users when they upgrade their kernel and find that it > >breaks Python, despite ABI compatibility promises. > > Indeed. It's small consolation that it would also break sendmail... > > >What code would have a hold of argv, except main() which just calls > >Py_Main()? > > In the normal python interpreter, nothing. My concern was more with a > system that embeddeds Python... Which is why I'd lean more towards an > implementation which required an explicit initialization call to enable. > That would signal Python that the embedder knew of and condoned the > modification of the process string. > > I'm torn between making it a build-time option in the SRPM (defaulting to > disabled) and just dropping it as "too contentious". Sadly, the least > objectional way to make it happen is probably making it os.pstat() and > wrapping the pstat() system call on the platforms that have it. "Sadly" > because Linux doesn't seem to be one of those platforms... Let's just drop it. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 9 15:27:49 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 10:27:49 -0400 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: Your message of "Tue, 09 Apr 2002 17:27:18 +1000." <200204090727.g397RIo03169@mbuna.arbhome.com.au> References: <200204090727.g397RIo03169@mbuna.arbhome.com.au> Message-ID: <200204091427.g39ERn920347@odiug.zope.com> > > > I also like Andrew's idea of making everybody commit their changes in > > > both branches -- to scale the effort of keeping the maintenance branch > > > up-to-date. > > > > For bug fixes, I encourage that. The question will come up whether new > > features also need to be back-ported. > > There's a couple of simple cases: > > . Obvious bugs that should be backported. Where possible, the original > committer should apply this. > > . Obvious new features that introduce new behaviour. This is something > that shouldn't be applied automatically, but _if_ the feature is something > that should be considered for the stable release then that should be > flagged in the comment, but not applied to the branch straight away. I just applied a patch to typeobject.c that makes the x[y] operation three times faster when type(x) is a subclass of list that doesn't override __getitem__. This isn't a bug, so I didn't even think of considering it for 2.2.1 -- but now I'm seriously rethinking that, and it will probably end up in 2.2.2, since it also doesn't introduce any incompatible behavior (assuming there aren't any bugs in the patch :-). But this is a serious amount of extra work -- as part of the change, I refactored some code, etc. --Guido van Rossum (home page: http://www.python.org/~guido/) From hughett@mercur.uphs.upenn.edu Tue Apr 9 15:32:49 2002 From: hughett@mercur.uphs.upenn.edu (Paul Hughett) Date: Tue, 9 Apr 2002 10:32:49 -0400 Subject: [Python-Dev] Release Schedules (was Stability & change) In-Reply-To: <3CB2479C.F5A475A4@metaslash.com> (message from Neal Norwitz on Mon, 08 Apr 2002 21:45:00 -0400) References: <3CB2479C.F5A475A4@metaslash.com> Message-ID: <200204091432.g39EWnc16164@mercur.uphs.upenn.edu> Neal wrote: > 'Major' releases (roughly corresponding to Linux kernel even releases) > would occur every ~ 18-36 months. These releases would be full > executable, doc, etc. This seems to be the crux of what many people > want. They want a vibrant changing language. But they don't want to > have to deal with that change. They want longer cycles. We are > talking about users of the language, not hard-core developers. These > releases would still go through the alpha, beta, gamma releases. The > last development release in a cycle would become the first alpha. > Bugfix or point releases (eg, 2.x.y) would be made as needed, ~ 3 > months. These releases would still go through the alpha, beta, gamma > releases. This release would be concurrent with the development > release. BINGO! That's what I want as a production user. Punctuated equilibrium, as the paleontologists call it. (I'd tighten up the interval to 12-18 months, but that's a quibble.) Paul Hughett From fdrake@acm.org Tue Apr 9 15:33:41 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 9 Apr 2002 10:33:41 -0400 Subject: [Python-Dev] 2.2.1 release mechanics In-Reply-To: <15538.62233.177329.83289@grendel.zope.com> References: <2m7knspj43.fsf@starship.python.net> <15538.1011.400324.537358@grendel.zope.com> <2mwuvhi88s.fsf@starship.python.net> <15538.62233.177329.83289@grendel.zope.com> Message-ID: <15538.64453.5323.939692@grendel.zope.com> Fred L. Drake, Jr. writes: > It won't take long, but I am looking at the patch to produce info > files. The patch just wasn't robust enough to pass through the emacs that came with RedHat 7.2, so I've rejected the patch. I'm building the 2.2.1 docs now; this will be a full set, not an abbreviated HTML-only release. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From fredrik@pythonware.com Tue Apr 9 15:34:50 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Tue, 9 Apr 2002 16:34:50 +0200 Subject: [Python-Dev] Re: Stability and change References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15538.194.350128.498326@12-248-41-177.client.attbi.com> <200204082310.g38NAUD14877@mercur.uphs.upenn.edu> <200204090019.g390J3C15946@pcp742651pcs.reston01.va.comcast.net> Message-ID: <043701c1dfd3$b7c5c690$0900a8c0@spiff> Guido wrote: > I'd be *very* surprised that a "mere" 10K lines of extension code > would require significant work to keep it working under 2.2. I don't > know where you get this idea, but Python's C API is actually quite > stable, and more often than not, no changes are necessary to keep > 1.5.2 code conformant with 2.2, or the changes are limited to 1 or 2 > lines per file. fwiw, I cannot recall ever having to change anything on the C API level when upgrading (and we're talking 100k's of lines here, starting at Python 1.2) downgrading is a different story (we generally support everything from 1.5.2 and upwards), but it usually only takes an #if PY_VERSION_HEX or two to fix that. From guido@python.org Tue Apr 9 15:36:04 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 10:36:04 -0400 Subject: [Python-Dev] Release Schedules (was Stability & change) In-Reply-To: Your message of "09 Apr 2002 09:31:53 +0200." References: <0204090852340H.14931@arthur> Message-ID: <200204091436.g39Ea4m20406@odiug.zope.com> > Alex Martelli writes: > > > Giving strip methods an optional arguments, the characters that > > are OK to throw away (stopping the stripping when a > > not-to-be-thrown- away character shows up -- defaulting to > > whitespace like now of course), would help here and also with > > other related issues such as emulating Perl's "chomp". I also > > think it's the kind of change that would be OK in a minor release > > in the stable branch. > > This is http://python.org/sf/444708. Which I'll reopen, because this is the first good use case for that suggestion I've seen. Alex is sounding much like me now (i.e., we've lured him over to our side :-): str() of some expressions will change, but it's "OK in a minor release" since we can provide a nice way to deal with the effect. Never mind that code that deals with this in the traditional way *will* break silently (displaying numbers that are a factor 10 too small!), which IMO is worse than the worst effect of str(1==1) changing (at least that'll fail with an error when it fails). Not that I disagree, but I expect that if this happens to break Paul Rubin's code (to use a random example :-), all hell will break loose. --Guido van Rossum (home page: http://www.python.org/~guido/) From niemeyer@conectiva.com Tue Apr 9 15:38:32 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Tue, 9 Apr 2002 11:38:32 -0300 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: <3CB2E244.6010102@strakt.com> References: <3CB2E244.6010102@strakt.com> Message-ID: <20020409113832.A990@ibook.distro.conectiva> Hi Jacob! > I would also like to add that there is a benefit with the multiple > numbering scheme. The various distribution makers will most probably > build packages for both the stable and the experimental versions, > allowing people to easily maintain both versions on their machines. [...] We are already doing this here. We have 2.1 packaged as an alternative package for 2.2, and they can be installed at the same time. OTOH, 2.2 is the main package, since it's passed the updates we've done in the various systems without any problem. 2.1 is not being distributed right now, because of that (even though we have the package, and use in some clients with Zope and alike). I wouldn't like to be forced to package 2.1 as the main stream package, because 2.2 is tagged as "unstable" (or "in-development", or whatever). > With the current situation, most distribution builders will skip the > alpha, beta and rc versions, thus decreasing the population that will > actually test them. The problem is not about the naming scheme ("alpha", "beta", etc), but its stability. I'm not going to include Python in an alpha stage, no matter how it's named. OTOH, I'd include an rc version as we know today, if the deadline obligate me to. Even with all these discussions, I don't have a favorable position to the change yet. That's probably because I've always had a terrible impression about the Linux release scheme. With API breaks at anytime and releases that just don't compile, I'd say Linux has two branches, UNSTABLE and VERY-UNSTABLE. At the same time, I've always had the python release scheme as a well designed one. I don't think changing it will help Python. Indeed, it may even help those missinformed heads create more confusion, and consume the already rare time of core developers. Best regards! -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From skip@pobox.com Tue Apr 9 15:41:57 2002 From: skip@pobox.com (Skip Montanaro) Date: Tue, 9 Apr 2002 09:41:57 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15538.4739.536592.90221@anthem.wooz.org> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> <15538.1515.580133.785503@12-248-41-177.client.attbi.com> <15538.1632.252404.964819@anthem.wooz.org> <15538.4237.873211.294420@12-248-41-177.client.attbi.com> <15538.4739.536592.90221@anthem.wooz.org> Message-ID: <15538.64949.734475.150359@12-248-41-177.client.attbi.com> SM> It works for 2.4.x for x >= 3 and for 2.6.x for x <= 9. ... SM> No support for any development branch releases (e.g. 2.N.x, where N SM> is odd) is implied. BAW> IMHO it will still confuse a lot of people. I don't think it's the BAW> same as the Linux kernel because I suspect that people upgrade BAW> their kernels a /lot/ less often then they'll upgrade their Python BAW> version. What's the version numbering scheme got to do with the rate at which people upgrade their local copy of Python? Here's my gut feeling on timing of releases. Typically, I would expect point (micro) releases on the experimental branch to occur about once a month, and probably be little more than adding a CVS tag and generating a source tarball. Hopefully, no release manager would be required. There should be few frantic checkins because if you miss this release, the next one's only a month or so away. On the stable branch, releases would come less often, perhaps as often as once every two months, but more likely every four to six months, whenever "enough" bug fixes had been applied or when a particularly nasty bug got fixed we want people to take advantage. The generation of such releases would be a bit more involved, but not terribly much. If you get rid of micro release numbers in non-code files (like the docs), I think you can disconnect the timing of code and documentation a bit. Splits of the experimental branch into new stable and experimental branches would probably occur about once every 12-18 months and be governed by the functional completeness of the experimental branch and rate of bug fixes applied to it. It might also be driven by new features you want to get in but down want to upset the apple cart so-to-speak. In that case I'd say you split, then apply the new features to the new experimental branch. Skip From aleax@aleax.it Tue Apr 9 15:41:49 2002 From: aleax@aleax.it (Alex Martelli) Date: Tue, 9 Apr 2002 16:41:49 +0200 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: <200204091427.g39ERn920347@odiug.zope.com> References: <200204090727.g397RIo03169@mbuna.arbhome.com.au> <200204091427.g39ERn920347@odiug.zope.com> Message-ID: On Tuesday 09 April 2002 04:27 pm, Guido van Rossum wrote: ... > I just applied a patch to typeobject.c that makes the x[y] operation > three times faster when type(x) is a subclass of list that doesn't > override __getitem__. This isn't a bug, so I didn't even think of > considering it for 2.2.1 -- but now I'm seriously rethinking that, and > it will probably end up in 2.2.2, since it also doesn't introduce any > incompatible behavior (assuming there aren't any bugs in the patch :-). *Applause*!!! This is EXACTLY the sort of thing I had in mind when I was talking about *actively supported* releases on the stable track. Exactly the kind of thing that gives managers that need to select a language a warm, fuzzy feeling ... ! *Bravo*! > But this is a serious amount of extra work -- as part of the change, I > refactored some code, etc. Sigh, I guess that's the downside:-(. Supporting stable-track releases say "this IS alive and actively supported" exactly BECAUSE it takes effort -- it's a message that validates itself as sincere. Alex From akuchlin@mems-exchange.org Tue Apr 9 15:47:18 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Tue, 9 Apr 2002 10:47:18 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204091420.g39EKPF16144@mercur.uphs.upenn.edu> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15538.194.350128.498326@12-248-41-177.client.attbi.com> <200204082310.g38NAUD14877@mercur.uphs.upenn.edu> <200204090019.g390J3C15946@pcp742651pcs.reston01.va.comcast.net> <200204091420.g39EKPF16144@mercur.uphs.upenn.edu> Message-ID: <20020409144717.GA7946@crystal.mems-exchange.org> On Tue, Apr 09, 2002 at 10:20:25AM -0400, Paul Hughett wrote: >version-to-version upgrades are generally pretty easy. And a detailed >list of the API changes made, so that I can review it and say, "No problem >there. Not there either. Oops, better fix that..." and be able to get >a reasonable estimate of the effort required for an upgrade. If you've Misc/NEWS in the source tree contains a list of C-level changes in the "C API" section. There's also a brief section titled "Interpreter Changes and Fixes" in the "What's New" document, but I don't try to be exhaustive, so the Misc/NEWS file is a better source. Side note: The NEWS file is split into separate parts for each alpha and beta release; at some point, should all the entries for 2.2a1, 2.2a2, 2.2b1, &c., be merged into a single section for 2.2final? Otherwise finding all the C API changes in 2.2 requires a lot of scrolling and hunting. --amk (www.amk.ca) "I would not tell them too much," said Holmes. "Women are never to be entirely trusted, -- not the best of them." -- From "The Sign of the Four" From skip@pobox.com Tue Apr 9 15:51:34 2002 From: skip@pobox.com (Skip Montanaro) Date: Tue, 9 Apr 2002 09:51:34 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <2mr8lpi4mr.fsf@starship.python.net> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> <15538.1515.580133.785503@12-248-41-177.client.attbi.com> <15538.1632.252404.964819@anthem.wooz.org> <15538.4237.873211.294420@12-248-41-177.client.attbi.com> <2mr8lpi4mr.fsf@starship.python.net> Message-ID: <15538.65526.586434.980032@12-248-41-177.client.attbi.com> >> Over time people should become aware that odd-numbered minor releases >> are always development releases and are not to be relied upon. Michael> In which case you might as well not release "odd-numbered minor Michael> releases" becuase 90+% of the people who would use them get Michael> Python from CVS anyway. You still need releases. I'm advocating a completely different way of deciding when to release, however. Instead of generating PEP 283 (Python 2.3 Release Schedule), the timing of when to split the experimental branch should be driven by completeness of the functional goals for that branch and the bug fix rate for that branch. This is not a commercial enterprise. I don't think there is a "market window" that has to be hit that should dictate, "we have to release 2.3 in mid-August, regardless of what else is going on." When you have a time-driven schedule, I think there is a temptation to find "something new" to put into every release. Measuring bug fix rate can be done directly by keeping historical information on the number of SF bugs closed per week (my little weekly script goes partway there now, I suppose). I suggest that the Release Schedule PEPs instead become Goal PEPs that record the goals for the next release and their status as well has bug fix information for the current micro release. Repeat after me: Experimental micro releases should be cheap. Skip From Anthony Baxter Tue Apr 9 15:50:05 2002 From: Anthony Baxter (Anthony Baxter) Date: Wed, 10 Apr 2002 00:50:05 +1000 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: Message from Alex Martelli of "Tue, 09 Apr 2002 16:41:49 +0200." <200204091441.g39EftT20949@valdez.interlink.com.au> Message-ID: <200204091450.g39Eo5J08465@mbuna.arbhome.com.au> >>> Alex Martelli wrote > On Tuesday 09 April 2002 04:27 pm, Guido van Rossum wrote: > > But this is a serious amount of extra work -- as part of the change, I > > refactored some code, etc. > Sigh, I guess that's the downside:-(. Supporting stable-track releases > say "this IS alive and actively supported" exactly BECAUSE it takes > effort -- it's a message that validates itself as sincere. That's where I see the patch czar role coming in - sometimes that's something that can be passed off to them to do, while someone like Guido can work on something more important, like an alternate to whitespace-based blocks, or whatever. Anthony Anthony -- Anthony Baxter It's never too late to have a happy childhood. From guido@python.org Tue Apr 9 15:49:05 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 10:49:05 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "09 Apr 2002 09:30:21 BST." <2mzo0di89u.fsf@starship.python.net> References: <2m7kni5rvk.fsf@starship.python.net> <200204081225.g38CPNn10186@mbuna.arbhome.com.au> <15537.56727.412033.844839@walden.zope.com> <200204082240.g38Me1n15697@pcp742651pcs.reston01.va.comcast.net> <20020409013917.GA3135@ute.mems-exchange.org> <2mzo0di89u.fsf@starship.python.net> Message-ID: <200204091449.g39En6V20509@odiug.zope.com> > It's not that hard to backport fixes. The trawling through logs and > regression testing is the hard bit, plus the release mechanics. What I fear most is the risk of missing an important fix. How are you ever gonna find it back? Not every fix is accompanied with a unittest. > Though abandoning sf will help here. That's a done deal AFAIC (for file releases). I'm waiting a few days to remove the past releases on SF, so links on mirrors don't go dead. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Tue Apr 9 15:57:02 2002 From: skip@pobox.com (Skip Montanaro) Date: Tue, 9 Apr 2002 09:57:02 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: <15538.8494.31517.706412@12-248-41-177.client.attbi.com> Message-ID: <15539.318.116820.301667@12-248-41-177.client.attbi.com> Skip> Is *-RELEASE a third branch, or is it what falls out the end of Skip> the *-STABLE pipe? Chris> RELEASE is actually just a tag on the RELENG branch of the tree. Chris> For example, FreeBSD has the following branches in CVS for 4.x: Chris> RELENG_4 Chris> RELENG_4_3 Chris> RELENG_4_4 Chris> RELENG_4_5 ... Chris> The following section of the handbook explains in some more Chris> detail about how the two branches progress through their life Chris> cycle: Chris> http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/current-stable.html I think that still leaves my question unanswered, it just rephrases it. If *-RELEASE are simply tags on the RELENG branches, how does the RELENG branch relate to the STABLE branches? I assume there is some forking or renaming that goes on. What are the criteria for deciding it's no longer just stable, but is now release quality? Skip From Anthony Baxter Tue Apr 9 15:55:46 2002 From: Anthony Baxter (Anthony Baxter) Date: Wed, 10 Apr 2002 00:55:46 +1000 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: Message from Michael Hudson of "09 Apr 2002 11:26:35 +0100." <2mk7rhi2w4.fsf@starship.python.net> Message-ID: <200204091455.g39Etk608602@mbuna.arbhome.com.au> >>> Michael Hudson wrote > What about recent pymalloc changes? They should yield some > performance enhancements, will make no difference to Python code, but > probably require cooperation from extension authors... Aren't they binary-incompatible? In which case, thanks but no thanks, for me binary compatibility is one of the fundamental parts of a minor release. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From guido@python.org Tue Apr 9 15:53:53 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 10:53:53 -0400 Subject: [Python-Dev] Developer resources (was Re: Stability and change) In-Reply-To: Your message of "09 Apr 2002 10:55:40 BST." <2mn0wdi4bn.fsf@starship.python.net> References: <2mn0wdi4bn.fsf@starship.python.net> Message-ID: <200204091453.g39Errt20573@odiug.zope.com> > Answer to most of these questions should be under > http://www.python.org/dev/ somewhere, I > guess. http://www.python.org/dev/tools.html has a few answers. Some > belong in the docs. So who's gonna update all that? Also, IMO http://python.sourceforge.net/ should be shut down or become a redirect to http://python.org/dev/. --Guido van Rossum (home page: http://www.python.org/~guido/) From aleax@aleax.it Tue Apr 9 15:58:38 2002 From: aleax@aleax.it (Alex Martelli) Date: Tue, 9 Apr 2002 16:58:38 +0200 Subject: [Python-Dev] Release Schedules (was Stability & change) In-Reply-To: <200204091436.g39Ea4m20406@odiug.zope.com> References: <200204091436.g39Ea4m20406@odiug.zope.com> Message-ID: On Tuesday 09 April 2002 04:36 pm, Guido van Rossum wrote: ... > > This is http://python.org/sf/444708. > > Which I'll reopen, because this is the first good use case for that > suggestion I've seen. > > Alex is sounding much like me now (i.e., we've lured him over to our > side :-): Nolo contendere -- I _was_ originally dubious on int/long unification (not negative, but dubious), but having tasted a little I do want more. > str() of some expressions will change, but it's "OK in a str(23**45) is '18956258430116202791319715713277227626159289499745290235663543' in Python 2.2; how will it change? I must be confused -- I thought repr() could yet change, while str() already had for the int/long case? > minor release" since we can provide a nice way to deal with the > effect. Never mind that code that deals with this in the traditional > way *will* break silently (displaying numbers that are a factor 10 too > small!), which IMO is worse than the worst effect of str(1==1) > changing (at least that'll fail with an error when it fails). Not Yes, I'll have to deal with the breakage in my old code (if/when I ever dust off my old bridge-research stuff, which I haven't had time to revisit in ages) -- I do have a few cases in which repr rather than str comes into play, assuming that's what's going to change (the str changes I had already dealt with, I think). (As str(somelist) uses repr on the items, I have this problem...). I guess the difference is that I see very different magnitude and opposite signs in the conceptual effects of the two changes -- introducing booleans vs hiding int/long distinctions. The latter means leaving the problems of "how many bits does this number have" to rare, specific low-level uses, just as it should be. The former "fills a badly needed gap" IMHO. And for the costs, changing repr() seems slightly lesser -- had str(1==1) remained as '1' while only repr changed, I'd have accounted that as a fractionally lower (practical) cost for PEP 285. So in the end it still comes down to total cost vs total benefit, where presumably reasonable people can differ on each of the two sides of the equation (and the decision is up to you), rather than absolute, objective rules. > that I disagree, but I expect that if this happens to break Paul > Rubin's code (to use a random example :-), all hell will break loose. Not everybody need be reasonable, of course -- being unreasonable used to be women's privilege, but we have equality of the sexes now:-). Alex From guido@python.org Tue Apr 9 15:59:26 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 10:59:26 -0400 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: Your message of "09 Apr 2002 11:26:35 BST." <2mk7rhi2w4.fsf@starship.python.net> References: <200204090727.g397RIo03169@mbuna.arbhome.com.au> <2mk7rhi2w4.fsf@starship.python.net> Message-ID: <200204091459.g39ExQ820648@odiug.zope.com> > What about recent pymalloc changes? They should yield some > performance enhancements, will make no difference to Python code, but > probably require cooperation from extension authors... That would be another Project. :-( May be worth it, may be not. I'd hate to foist pymalloc on the community in 2.2.2 only to find out that it breaks in 1% of situations. It does suggest that we should try to strive for a malloc API that doesn't require changes to extensions written (according to the docs) for 2.2. --Guido van Rossum (home page: http://www.python.org/~guido/) From aleax@aleax.it Tue Apr 9 16:01:35 2002 From: aleax@aleax.it (Alex Martelli) Date: Tue, 9 Apr 2002 17:01:35 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15538.65526.586434.980032@12-248-41-177.client.attbi.com> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <2mr8lpi4mr.fsf@starship.python.net> <15538.65526.586434.980032@12-248-41-177.client.attbi.com> Message-ID: On Tuesday 09 April 2002 04:51 pm, Skip Montanaro wrote: > >> Over time people should become aware that odd-numbered minor > >> releases are always development releases and are not to be relied > >> upon. > > Michael> In which case you might as well not release "odd-numbered > minor Michael> releases" becuase 90+% of the people who would use them get > Michael> Python from CVS anyway. > > You still need releases. I'm advocating a completely different way of > deciding when to release, however. Instead of generating PEP 283 (Python > 2.3 Release Schedule), the timing of when to split the experimental branch > should be driven by completeness of the functional goals for that branch > and the bug fix rate for that branch. This is not a commercial enterprise. I find this line of reasoning VERY convincing -- I hadn't thought of it myself, but you make me wish I had:-). > I don't think there is a "market window" that has to be hit that should > dictate, "we have to release 2.3 in mid-August, regardless of what else is > going on." When you have a time-driven schedule, I think there is a > temptation to find "something new" to put into every release. Well put! Goal-driven releases, rather than time-driven ones, seem important for both tracks, the stable and the experimental one. Alex From akuchlin@mems-exchange.org Tue Apr 9 16:06:58 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Tue, 9 Apr 2002 11:06:58 -0400 Subject: [Python-Dev] 2.3 goals In-Reply-To: References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <2mr8lpi4mr.fsf@starship.python.net> <15538.65526.586434.980032@12-248-41-177.client.attbi.com> Message-ID: <20020409150657.GA8019@crystal.mems-exchange.org> On Tue, Apr 09, 2002 at 05:01:35PM +0200, Alex Martelli wrote: >Well put! Goal-driven releases, rather than time-driven ones, seem >important for both tracks, the stable and the experimental one. So, does anyone want to set a list of goals for 2.3? Here's my list: Implement PEP 262, and a simple package manager on top of it. Finish the block encryption PEP. Neil S. suggested to me that adding AES to Python's standard library would be a good idea. (And maybe deprecate rotor if we do that.) Add support for the long-awaited Python catalog. Documentation: complete the distribution and installation manuals Look over the Demos/ directory and update where required. (Of course, I wanted to do that for every previous 2.x release, too, so don't be too surprised if it doesn't happen.) --amk (www.amk.ca) As soon as I found myself in this horrible place, I started looking for you. -- Susan, in "The Five Doctors" From guido@python.org Tue Apr 9 16:04:44 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 11:04:44 -0400 Subject: [Python-Dev] Multiple revisions [was: Re: Stability and change] In-Reply-To: Your message of "Tue, 09 Apr 2002 13:35:58 +0200." <3CB2D21E.1000804@strakt.com> References: <3CB2D21E.1000804@strakt.com> Message-ID: <200204091504.g39F4j620711@odiug.zope.com> > You can significantly reduce the time needed to apply patches to several > versions of your software by using a smart revision control system. > > I recently discovered "arch", which has been designed to handle multiple > parallell versions. It is a marvel, and it comes with really good > descriptions of how to handle the problem in practice. I know that > Linus Torvalds is considering it for the Linux kernel. > > http://regexps.com/ If I had to start over, I'd be happy to consider arch (and Perforce, and Bitkeeper, and, oh, why not, Subversion). As it is, this would mean ripping out the foundation from our development process -- everything we do is tied to CVS on SF. So, unless arch runs on top of an existing CVS setup, too bad. --Guido van Rossum (home page: http://www.python.org/~guido/) From pobrien@orbtech.com Tue Apr 9 16:17:06 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Tue, 9 Apr 2002 10:17:06 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204091449.g39En6V20509@odiug.zope.com> Message-ID: [Guido van Rossum] > > What I fear most is the risk of missing an important fix. How are you > ever gonna find it back? Not every fix is accompanied with a unittest. I think more emphasis should be placed on creating unit tests. While the number of unit tests is growing, achieving some level of completeness should really help with the stability and change issues and the public perception of Python's stability, IMHO. The reason I bring this up is that the creation of unit tests is something that can be done by volunteers (such as myself) who aren't capable of messing with the actual Python C code but want to help with Python where they can. Retrofitting existing code with unit tests is a pretty tedious pain in the butt. I just did it for a bunch of my own code. Having gotten over the hump, I'm sold on the benefits of unit testing. So I'd like to see all the gaps in Python's own test suite filled in. While I'm pretty overworked at the moment, as soon as I have some free time I'm willing to organize an effort to fill in the gaps in the existing suite of unit tests, if this kind of effort is deemed appropriate. What I think is needed is a list of what is missing, a way to coordinate the check in of new tests, some documentation to make it easier to get over the learning curve on unittest, and someone to coordinate the volunteers. Is this worth pursuing? I'm willing to roll up my sleeves and pitch in if it is. (Please use the new boolean type when replying. ) --- Patrick K. O'Brien Orbtech From Anthony Baxter Tue Apr 9 16:19:29 2002 From: Anthony Baxter (Anthony Baxter) Date: Wed, 10 Apr 2002 01:19:29 +1000 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Message from "Patrick K. O'Brien" of "Tue, 09 Apr 2002 10:17:06 EST." Message-ID: <200204091519.g39FJT309099@mbuna.arbhome.com.au> >>> "Patrick K. O'Brien" wrote > I think more emphasis should be placed on creating unit tests. While the > number of unit tests is growing, achieving some level of completeness should > really help with the stability and change issues and the public perception > of Python's stability, IMHO. +True Anthony From mal@lemburg.com Tue Apr 9 16:24:13 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 09 Apr 2002 17:24:13 +0200 Subject: [Python-Dev] 2.3 goals References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <2mr8lpi4mr.fsf@starship.python.net> <15538.65526.586434.980032@12-248-41-177.client.attbi.com> <20020409150657.GA8019@crystal.mems-exchange.org> Message-ID: <3CB3079D.AAE79437@lemburg.com> Andrew Kuchling wrote: >=20 > On Tue, Apr 09, 2002 at 05:01:35PM +0200, Alex Martelli wrote: > >Well put! Goal-driven releases, rather than time-driven ones, seem > >important for both tracks, the stable and the experimental one. >=20 > So, does anyone want to set a list of goals for 2.3? Here's my list: >=20 > Implement PEP 262, and a simple package manager on top of it. >=20 > Finish the block encryption PEP. Neil S. suggested to me that > adding AES to Python's standard library would be a good idea. > (And maybe deprecate rotor if we do that.) I don't think it's a good idea to add strong crypto to Python's core. It's better to ship it as standard add-on (we'll probably have to do the same with the Asian codec package). Since python.org is hosted in the Netherlands, you only have to export the crypto there and then post the apporpriate notice to the BXA -- this makes it much easier on behalf of the PSF to get this right in the light of the US export=20 rules. =20 > Add support for the long-awaited Python catalog. >=20 > Documentation: complete the distribution and installation manuals >=20 > Look over the Demos/ directory and update where required. (Of > course, I wanted to do that for every previous 2.x release, too, so > don't be too surprised if it doesn't happen.) + Implement phase 1 of the Python source code encoding PEP (Martin has already done a lot of work in this area). + Add Walter D=F6rwald's codec error handling patch (provided he finishes it in time). =20 --=20 Marc-Andre Lemburg ________________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From akuchlin@mems-exchange.org Tue Apr 9 16:28:42 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Tue, 9 Apr 2002 11:28:42 -0400 Subject: [Python-Dev] Developer resources (was Re: Stability and change) In-Reply-To: <200204091453.g39Errt20573@odiug.zope.com> References: <2mn0wdi4bn.fsf@starship.python.net> <200204091453.g39Errt20573@odiug.zope.com> Message-ID: <20020409152842.GA8225@crystal.mems-exchange.org> On Tue, Apr 09, 2002 at 10:53:53AM -0400, Guido van Rossum wrote: >Also, IMO http://python.sourceforge.net/ should be shut down or become >a redirect to http://python.org/dev/. All of the content on python.sourceforge.net now seems to be under python.org/dev, *except* for the daily snapshots of the CVS tree, which are still at python.sf.net/snapshots/ . So a redirect for the top-level page can now be set up. --amk (www.amk.ca) "Arrest you!" said Holmes. "This is really most grati-- most interesting." -- Sherlock Holmes, in "The Adventure of the Norwood Builder" From neal@metaslash.com Tue Apr 9 16:31:08 2002 From: neal@metaslash.com (Neal Norwitz) Date: Tue, 09 Apr 2002 11:31:08 -0400 Subject: [Python-Dev] Re: Stability and change References: Message-ID: <3CB3093C.B7A22727@metaslash.com> "Patrick K. O'Brien" wrote: > What I think is needed is a list of what is missing The list is quite large. Skip the rest of this messge for those who are uninterested. The list contains both python modules as well as C modules. Please let me know about mistakes. Neal -- C Modules: ---------- _curses_panel flmodule fmmodule fpectlmodule fpetestmodule getpath pcremodule posixmodule pypcre readline regexpr resource sgimodule svmodule syslogmodule tclNotify termios tkappinit _tkinter yuvconvert Python Modules: --------------- aifc.py asyncore.py audiodev.py base64.py BaseHTTPServer.py bdb.py CGIHTTPServer.py cgitb.py chunk.py cmd.py code.py colorsys.py dbhash.py dis.py filecmp.py formatter.py ftplib.py getpass.py gopherlib.py htmlentitydefs.py ihooks.py imghdr.py imputil.py keyword.py knee.py linecache.py macpath.py macurl2path.py mailcap.py markupbase.py mimify.py multifile.py mutex.py nntplib.py nturl2path.py os2emxpath.py pdb.py pipes.py poplib.py posixfile.py pre.py pstats.py py_compile.py pydoc.py Queue.py random.py reconvert.py regex_syntax.py regsub.py rexec.py rlcompleter.py sched.py shelve.py shlex.py shutil.py SimpleHTTPServer.py SimpleXMLRPCServer.py site.py smtpd.py smtplib.py sndhdr.py statcache.py stringold.py sunaudio.py sunau.py symbol.py tabnanny.py telnetlib.py toaiff.py tty.py UserDict.py UserList.py user.py UserString.py warnings.py webbrowser.py whichdb.py whrandom.py xdrlib.py xmlrpclib.py From neal@metaslash.com Tue Apr 9 16:36:40 2002 From: neal@metaslash.com (Neal Norwitz) Date: Tue, 09 Apr 2002 11:36:40 -0400 Subject: [Python-Dev] 2.3 goals References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <2mr8lpi4mr.fsf@starship.python.net> <15538.65526.586434.980032@12-248-41-177.client.attbi.com> <20020409150657.GA8019@crystal.mems-exchange.org> Message-ID: <3CB30A88.F5CC1470@metaslash.com> Andrew Kuchling wrote: > > So, does anyone want to set a list of goals for 2.3? Here's my list: > > Implement PEP 262, and a simple package manager on top of it. > > Finish the block encryption PEP. Neil S. suggested to me that > adding AES to Python's standard library would be a good idea. > (And maybe deprecate rotor if we do that.) > > Add support for the long-awaited Python catalog. > > Documentation: complete the distribution and installation manuals > > Look over the Demos/ directory and update where required. (Of > course, I wanted to do that for every previous 2.x release, too, so > don't be too surprised if it doesn't happen.) * New tests * Fix doc bugs on SF (I know Fred needs help with this) * Remove use of any deprecated features in the core * Doc deprecated features appropriately * Move deprecated features under Py_DEPRECATED (or whatever is decided) * Deprecate modules which are unmaintained, or perhaps make a new category for modules 'Unmaintained' In general, lots of cleanup so it is easier to move forward. I've been trying to do these things. But since I'm new, I don't know about most of the baggage. Neal From thomas.heller@ion-tof.com Tue Apr 9 16:37:51 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Tue, 9 Apr 2002 17:37:51 +0200 Subject: [Python-Dev] Re: Stability and change References: <3CB3093C.B7A22727@metaslash.com> Message-ID: <091101c1dfdc$82ad01e0$e000a8c0@thomasnotebook> From: "Neal Norwitz" > "Patrick K. O'Brien" wrote: > > > What I think is needed is a list of what is missing > > The list is quite large. Skip the rest of this messge for those > who are uninterested. The list contains both python modules > as well as C modules. Please let me know about mistakes. > > Neal > -- distutils is missing completely. I'm currently working on a test_distutils script which just compiles an extension module (src/Modules/xxmodule.c) and builds various distribution formats. Better than nothing, IMO. Thomas From skip@pobox.com Tue Apr 9 17:09:37 2002 From: skip@pobox.com (Skip Montanaro) Date: Tue, 9 Apr 2002 11:09:37 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204091420.g39EKPF16144@mercur.uphs.upenn.edu> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15538.194.350128.498326@12-248-41-177.client.attbi.com> <200204082310.g38NAUD14877@mercur.uphs.upenn.edu> <200204090019.g390J3C15946@pcp742651pcs.reston01.va.comcast.net> <200204091420.g39EKPF16144@mercur.uphs.upenn.edu> Message-ID: <15539.4673.910115.885056@12-248-41-177.client.attbi.com> Paul> Now I have to confess that, while I've downloaded the Paul> 2.0 source package, I've been too busy actually using Python to Paul> open it up and look at the API documentation. You mean 2.1 (2.1.3 was just released) or 2.2 (2.2.1 is about to be released), right? Skip From fredrik@pythonware.com Tue Apr 9 17:13:04 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Tue, 9 Apr 2002 18:13:04 +0200 Subject: [Python-Dev] Re: PEP Politics (was Re: Re: Stability and change) References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <15537.11319.576212.846144@anthem.wooz.org> <15537.15490.631372.551394@12-248-41-177.client.attbi.com> <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> Message-ID: <010a01c1dfe1$c177e3e0$0900a8c0@spiff> guido wrote: > And I'm saying that with the current user base, *anything* from me > will cause outrage. I don't think the comp.lang.python crowd represents a large part of the current user base... (most python programmers I know don't read any python mailing list or newsgroup -- they have better things to do ;-) From fredrik@pythonware.com Tue Apr 9 17:15:21 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Tue, 9 Apr 2002 18:15:21 +0200 Subject: [Python-Dev] Re: Stability and change References: <02040822525808.14931@arthur> <200204082100.g38L0ol15046@pcp742651pcs.reston01.va.comcast.net> <0204082301000A.14931@arthur> <200204082141.g38Lfp015226@pcp742651pcs.reston01.va.comcast.net> Message-ID: <010b01c1dfe1$c17dfe60$0900a8c0@spiff> guido wrote: > > > Tell that to the people who write L.append(1, 2, 3) despite that = it > > > was never documented. :-( > >=20 > > I do -- if and when I see that (haven't seen it in ages). Technical > > managers at development shops know about stuff "accidentally > > working" and are more tolerant of bug-fixes against that than of > > other backwards-compatibility breaks, in my experience. >=20 > Hm. This was widely complained about, probably because many examples > (including in books) got it wrong. fwiw, these days, I get TypeError: append() takes exactly 1 argument (2 given) far too often to think that I picked this up by reading the wrong books. it's an easy mistake, and it used to do the "right thing"... From guido@python.org Tue Apr 9 17:28:25 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 12:28:25 -0400 Subject: [Python-Dev] Re: PEP Politics (was Re: Re: Stability and change) In-Reply-To: Your message of "Tue, 09 Apr 2002 18:13:04 +0200." <010a01c1dfe1$c177e3e0$0900a8c0@spiff> References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <15537.11319.576212.846144@anthem.wooz.org> <15537.15490.631372.551394@12-248-41-177.client.attbi.com> <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> Message-ID: <200204091628.g39GSQf22128@odiug.zope.com> > I don't think the comp.lang.python crowd represents a large part > of the current user base... Maybe I should stop posting PEPs to c.l.py, and tell them that if they want a say in the future of the language, they should subscribe to python-dev? --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Tue Apr 9 17:30:32 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 9 Apr 2002 12:30:32 -0400 Subject: [Python-Dev] 2.2.1 docs Message-ID: <15539.5928.855738.194157@grendel.zope.com> The Python 2.2.1 docs have now been published on python.org, and are listed as the "current" documentation. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From sholden@holdenweb.com Tue Apr 9 17:28:14 2002 From: sholden@holdenweb.com (Steve Holden) Date: Tue, 9 Apr 2002 12:28:14 -0400 Subject: [Python-Dev] Developer resources (was Re: Stability and change) References: <200204081528.g38FSi112614@mbuna.arbhome.com.au><200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net><20020408163903.GA2216@crystal.mems-exchange.org><200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net><15537.55112.191870.654238@anthem.wooz.org><15538.1515.580133.785503@12-248-41-177.client.attbi.com><15538.2623.323978.125527@slothrop.zope.com><20020408223655.GD562@panix.com> <15538.32008.560965.639327@grendel.zope.com> Message-ID: <006f01c1dfe3$8ce6c9f0$8001a8c0@COMPUTER> Fred Drake wrote: > > (BTW, is the Starship membership closed now? If so, the starship home > page should be updated.) > Well, certainly the starship and (earlier) the PSA were indistinguishable from /dev/null from the point of view of a relative newcomer (who had paid for a conference without attending it) wanting a starship account. regards Steve From akuchlin@mems-exchange.org Tue Apr 9 17:39:11 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Tue, 9 Apr 2002 12:39:11 -0400 Subject: [Python-Dev] 2.3 goals In-Reply-To: <20020409150657.GA8019@crystal.mems-exchange.org> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <2mr8lpi4mr.fsf@starship.python.net> <15538.65526.586434.980032@12-248-41-177.client.attbi.com> <20020409150657.GA8019@crystal.mems-exchange.org> Message-ID: <20020409163911.GB8520@crystal.mems-exchange.org> Oh, and another possible 2.3 thing: is it time to drop the PCRE code? --amk From fdrake@acm.org Tue Apr 9 17:41:50 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 9 Apr 2002 12:41:50 -0400 Subject: [Python-Dev] Developer resources (was Re: Stability and change) In-Reply-To: <006f01c1dfe3$8ce6c9f0$8001a8c0@COMPUTER> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> <15538.1515.580133.785503@12-248-41-177.client.attbi.com> <15538.2623.323978.125527@slothrop.zope.com> <20020408223655.GD562@panix.com> <15538.32008.560965.639327@grendel.zope.com> <006f01c1dfe3$8ce6c9f0$8001a8c0@COMPUTER> Message-ID: <15539.6606.780921.399696@grendel.zope.com> Steve Holden writes: > Well, certainly the starship and (earlier) the PSA were indistinguishable > from /dev/null from the point of view of a relative newcomer (who had paid > for a conference without attending it) wanting a starship account. Ouch! I was wondering since I've seen a few notices to the "crew" list from the starship admins over the last couple of months about various bits of software being updated, and wasn't sure if it was picking up or not. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From aahz@pythoncraft.com Tue Apr 9 17:42:26 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 9 Apr 2002 12:42:26 -0400 Subject: [Python-Dev] Re: PEP Politics (was Re: Re: Stability and change) In-Reply-To: <200204091628.g39GSQf22128@odiug.zope.com> References: <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> Message-ID: <20020409164226.GA23601@panix.com> On Tue, Apr 09, 2002, Guido van Rossum wrote: > > > I don't think the comp.lang.python crowd represents a large part > > of the current user base... > > Maybe I should stop posting PEPs to c.l.py, and tell them that if they > want a say in the future of the language, they should subscribe to > python-dev? Do you really want Paul Rubin on this mailing list? How about at least trying my suggestion that wording a PEP differently might make a difference? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From DavidA@ActiveState.com Tue Apr 9 17:42:59 2002 From: DavidA@ActiveState.com (David Ascher) Date: Tue, 09 Apr 2002 09:42:59 -0700 Subject: [Python-Dev] Re: Stability and change References: Message-ID: <3CB31A13.EF1BB615@activestate.com> On the topic of unit tests, I wonder if it makes sense to consider a new test directory outside of Lib/. As the test suite is beefed up (something which I think is a great idea), the disk space requirement for the test suite shouldn't be a factor -- the more non-overlapping tests the better, period -- but if it's part of the library tree, space _should_ be a concern because that stuff gets shipped to the millions of users. In other words, I'd like to have a test suite outside of what gets installed. The reason for not moving all of it out is that there is value in being able to run a post-install test suite, but I don't think it needs to be all of the tests. Reactions? --da From akuchlin@mems-exchange.org Tue Apr 9 17:45:01 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Tue, 9 Apr 2002 12:45:01 -0400 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: <200204091628.g39GSQf22128@odiug.zope.com> References: <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> Message-ID: <20020409164501.GD8520@crystal.mems-exchange.org> On Tue, Apr 09, 2002 at 12:28:25PM -0400, Guido van Rossum wrote: >Maybe I should stop posting PEPs to c.l.py, and tell them that if they >want a say in the future of the language, they should subscribe to >python-dev? NOOOOOO!!!!! I don't want to have to start using a killfile on python-dev traffic to eliminate endless windy threads that never go anywhere. --amk PS: NOOOOOO!!!!! From mal@lemburg.com Tue Apr 9 17:52:59 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 09 Apr 2002 18:52:59 +0200 Subject: [Python-Dev] Re: PEP Politics References: <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> Message-ID: <3CB31C6B.5A2FE5D@lemburg.com> Andrew Kuchling wrote: > > On Tue, Apr 09, 2002 at 12:28:25PM -0400, Guido van Rossum wrote: > >Maybe I should stop posting PEPs to c.l.py, and tell them that if they > >want a say in the future of the language, they should subscribe to > >python-dev? > > NOOOOOO!!!!! I don't want to have to start using a killfile on > python-dev traffic to eliminate endless windy threads that never go > anywhere. > > --amk > > PS: NOOOOOO!!!!! PPS: NOOOOOO!!!!! I'd suggest to either leave the discussions on c.l.p or setup a new mailing list (e.g. peps@python.org) for PEP postings. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From guido@python.org Tue Apr 9 18:13:58 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 13:13:58 -0400 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: Your message of "Tue, 09 Apr 2002 12:45:01 EDT." <20020409164501.GD8520@crystal.mems-exchange.org> References: <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> Message-ID: <200204091713.g39HDwf22586@odiug.zope.com> > On Tue, Apr 09, 2002 at 12:28:25PM -0400, Guido van Rossum wrote: > >Maybe I should stop posting PEPs to c.l.py, and tell them that if they > >want a say in the future of the language, they should subscribe to > >python-dev? > > NOOOOOO!!!!! I don't want to have to start using a killfile on > python-dev traffic to eliminate endless windy threads that never go > anywhere. > > --amk > > PS: NOOOOOO!!!!! I wasn't serious, but AFAICT /F was serious in his suggestion not to discuss my PEPs on c.l.py any more. I think the consequence of that would be that my PEPs only get discussed on python-dev. Then I see two possibilities: simply ignore the c.l.py crowd, or inviting them. I see serious downsides to each. So that means I'll have to face c.l.py every time I have a PEP, or stop writing PEPs. The latter sounds the most attractive -- I'll just check things in after a brief discussion on python-dev. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 9 18:19:18 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 13:19:18 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Tue, 09 Apr 2002 09:42:59 PDT." <3CB31A13.EF1BB615@activestate.com> References: <3CB31A13.EF1BB615@activestate.com> Message-ID: <200204091719.g39HJIE22614@odiug.zope.com> > On the topic of unit tests, I wonder if it makes sense to consider a new > test directory outside of Lib/. As the test suite is beefed up > (something which I think is a great idea), the disk space requirement > for the test suite shouldn't be a factor -- the more non-overlapping > tests the better, period -- but if it's part of the library tree, space > _should_ be a concern because that stuff gets shipped to the millions of > users. > > In other words, I'd like to have a test suite outside of what gets > installed. The reason for not moving all of it out is that there is > value in being able to run a post-install test suite, but I don't think > it needs to be all of the tests. > > Reactions? > > --da Disk space is cheap. YAGNI. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 9 18:21:32 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 13:21:32 -0400 Subject: [Python-Dev] Developer resources (was Re: Stability and change) In-Reply-To: Your message of "Tue, 09 Apr 2002 12:41:50 EDT." <15539.6606.780921.399696@grendel.zope.com> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> <15538.1515.580133.785503@12-248-41-177.client.attbi.com> <15538.2623.323978.125527@slothrop.zope.com> <20020408223655.GD562@panix.com> <15538.32008.560965.639327@grendel.zope.com> <006f01c1dfe3$8ce6c9f0$8001a8c0@COMPUTER> <15539.6606.780921.399696@grendel.zope.com> Message-ID: <200204091721.g39HLWm22627@odiug.zope.com> > Steve Holden writes: > > Well, certainly the starship and (earlier) the PSA were indistinguishable > > from /dev/null from the point of view of a relative newcomer (who had paid > > for a conference without attending it) wanting a starship account. > > Ouch! > > I was wondering since I've seen a few notices to the "crew" list from > the starship admins over the last couple of months about various bits > of software being updated, and wasn't sure if it was picking up or > not. > > -Fred The problem with starship is that there are no criteria to admit new users. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 9 18:23:08 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 13:23:08 -0400 Subject: [Python-Dev] 2.3 goals In-Reply-To: Your message of "Tue, 09 Apr 2002 12:39:11 EDT." <20020409163911.GB8520@crystal.mems-exchange.org> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <2mr8lpi4mr.fsf@starship.python.net> <15538.65526.586434.980032@12-248-41-177.client.attbi.com> <20020409150657.GA8019@crystal.mems-exchange.org> <20020409163911.GB8520@crystal.mems-exchange.org> Message-ID: <200204091723.g39HN8M22644@odiug.zope.com> > Oh, and another possible 2.3 thing: is it time to drop the PCRE code? Yes please, and anything that uses it (like pre.py). I guess sre.py must remain as mostly an alias for re.py. --Guido van Rossum (home page: http://www.python.org/~guido/) From DavidA@ActiveState.com Tue Apr 9 18:28:02 2002 From: DavidA@ActiveState.com (David Ascher) Date: Tue, 09 Apr 2002 10:28:02 -0700 Subject: [Python-Dev] Re: Stability and change References: <3CB31A13.EF1BB615@activestate.com> <200204091719.g39HJIE22614@odiug.zope.com> Message-ID: <3CB324A2.9317F3E1@activestate.com> Guido van Rossum wrote: > Disk space is cheap. YAGNI. Just FYI, I'm not concerned about disk space, more download bandwidth and what that means for people with small pipes. YAGNI probably still applies, and when it stops applying, we can deal with it. --da From aahz@pythoncraft.com Tue Apr 9 18:29:09 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 9 Apr 2002 13:29:09 -0400 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: <200204091713.g39HDwf22586@odiug.zope.com> References: <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <200204091713.g39HDwf22586@odiug.zope.com> Message-ID: <20020409172909.GA4657@panix.com> On Tue, Apr 09, 2002, Guido van Rossum wrote: > > I wasn't serious, but AFAICT /F was serious in his suggestion not to > discuss my PEPs on c.l.py any more. I think the consequence of that > would be that my PEPs only get discussed on python-dev. Then I see > two possibilities: simply ignore the c.l.py crowd, or inviting them. > I see serious downsides to each. So that means I'll have to face > c.l.py every time I have a PEP, or stop writing PEPs. The latter > sounds the most attractive -- I'll just check things in after a brief > discussion on python-dev. Seems to me that you're being unreasonably, er, boolean in your thinking here. From my POV, the fact that the BDFL follows the same PEP process as everyone else is a powerful point in favor of stability. There may not be a way to do it, but I'd sure like to try some things to lessen the difficulty of using c.l.py. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From akuchlin@mems-exchange.org Tue Apr 9 18:34:49 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Tue, 9 Apr 2002 13:34:49 -0400 Subject: [Python-Dev] 2.3 goals In-Reply-To: <200204091723.g39HN8M22644@odiug.zope.com> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <2mr8lpi4mr.fsf@starship.python.net> <15538.65526.586434.980032@12-248-41-177.client.attbi.com> <20020409150657.GA8019@crystal.mems-exchange.org> <20020409163911.GB8520@crystal.mems-exchange.org> <200204091723.g39HN8M22644@odiug.zope.com> Message-ID: <20020409173449.GA8981@crystal.mems-exchange.org> On Tue, Apr 09, 2002 at 01:23:08PM -0400, Guido van Rossum wrote: >> Oh, and another possible 2.3 thing: is it time to drop the PCRE code? > >Yes please, and anything that uses it (like pre.py). I guess sre.py >must remain as mostly an alias for re.py. Drop or deprecate? pre is mentioned as an implementation detail in the docs for the 're' module, so it could be argued that pre.py should trigger a warning in 2.3 and only be dropped in 2.4. (My inclination is to just delete it, though; 'pre' never supported Unicode and I doubt many people go out of their way to use it.) --amk (www.amk.ca) "Such minds, Hinton, shaping infinity itself." -- Gull, in FROM HELL #2 From fredrik@pythonware.com Tue Apr 9 18:39:40 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Tue, 9 Apr 2002 19:39:40 +0200 Subject: [Python-Dev] Re: PEP Politics References: <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <200204091713.g39HDwf22586@odiug.zope.com> Message-ID: <026601c1dfed$884808f0$ced241d5@hagrid> guido wrote: > I wasn't serious, but AFAICT /F was serious in his suggestion not to > discuss my PEPs on c.l.py any more. not sure that was what I suggested, really -- my point is that you cannot treat c.l.py as a representative subset of the current user base. most people are elsewhere, and most users won't see the PEP at all, no matter where you post it. From fdrake@acm.org Tue Apr 9 18:44:19 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 9 Apr 2002 13:44:19 -0400 Subject: [Python-Dev] 2.3 goals In-Reply-To: <200204091723.g39HN8M22644@odiug.zope.com> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <2mr8lpi4mr.fsf@starship.python.net> <15538.65526.586434.980032@12-248-41-177.client.attbi.com> <20020409150657.GA8019@crystal.mems-exchange.org> <20020409163911.GB8520@crystal.mems-exchange.org> <200204091723.g39HN8M22644@odiug.zope.com> <20020409173449.GA8981@crystal.mems-exchange.org> Message-ID: <15539.10355.794320.234469@grendel.zope.com> Guido van Rossum writes: > Yes please, and anything that uses it (like pre.py). I guess sre.py > must remain as mostly an alias for re.py. Andrew Kuchling writes: > Drop or deprecate? pre is mentioned as an implementation detail in > the docs for the 're' module, so it could be argued that pre.py should Mention as an implementation detail should not cause something to be kept around. I don't recall "sre" being anything other than an implementation detail, either, so I'm not convinced that name needs to be maintained. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From fdrake@acm.org Tue Apr 9 18:46:33 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 9 Apr 2002 13:46:33 -0400 Subject: [Python-Dev] Developer resources (was Re: Stability and change) In-Reply-To: <200204091721.g39HLWm22627@odiug.zope.com> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <200204081630.g38GUAC12764@pcp742651pcs.reston01.va.comcast.net> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> <15538.1515.580133.785503@12-248-41-177.client.attbi.com> <15538.2623.323978.125527@slothrop.zope.com> <20020408223655.GD562@panix.com> <15538.32008.560965.639327@grendel.zope.com> <006f01c1dfe3$8ce6c9f0$8001a8c0@COMPUTER> <15539.6606.780921.399696@grendel.zope.com> <200204091721.g39HLWm22627@odiug.zope.com> Message-ID: <15539.10489.290501.784475@grendel.zope.com> Guido van Rossum writes: > The problem with starship is that there are no criteria to admit new > users. I'm sure CNRI would be glad to receive a few PSA donations. ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From niemeyer@conectiva.com Tue Apr 9 18:47:40 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Tue, 9 Apr 2002 14:47:40 -0300 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: <200204091713.g39HDwf22586@odiug.zope.com> References: <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <200204091713.g39HDwf22586@odiug.zope.com> Message-ID: <20020409144740.A4422@ibook.distro.conectiva> [...] > I see serious downsides to each. So that means I'll have to face > c.l.py every time I have a PEP, or stop writing PEPs. The latter > sounds the most attractive -- I'll just check things in after a brief > discussion on python-dev. There's another alternative: you could just ask for someone else to *publish* your PEPs. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From guido@python.org Tue Apr 9 18:49:12 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 13:49:12 -0400 Subject: [Python-Dev] 2.3 goals In-Reply-To: Your message of "Tue, 09 Apr 2002 13:44:19 EDT." <15539.10355.794320.234469@grendel.zope.com> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <2mr8lpi4mr.fsf@starship.python.net> <15538.65526.586434.980032@12-248-41-177.client.attbi.com> <20020409150657.GA8019@crystal.mems-exchange.org> <20020409163911.GB8520@crystal.mems-exchange.org> <200204091723.g39HN8M22644@odiug.zope.com> <20020409173449.GA8981@crystal.mems-exchange.org> <15539.10355.794320.234469@grendel.zope.com> Message-ID: <200204091749.g39HnC722828@odiug.zope.com> > Mention as an implementation detail should not cause something to be > kept around. I don't recall "sre" being anything other than an > implementation detail, either, so I'm not convinced that name needs to > be maintained. There *are* people who import sre or pre because they think they have a preference. I propose to lose PCRE, but keep sre and pre as (deprecated) aliases for re. I.e. e.g. # pre.py import warning as _warning _warning.warn(..., "please use re, not pre") from re import * And ditto for sre. (Except maybe sre is an implementation artefact that may be harder to get rid of -- I'm not in a hurry there.) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 9 18:50:50 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 13:50:50 -0400 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: Your message of "Tue, 09 Apr 2002 19:39:40 +0200." <026601c1dfed$884808f0$ced241d5@hagrid> References: <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <200204091713.g39HDwf22586@odiug.zope.com> <026601c1dfed$884808f0$ced241d5@hagrid> Message-ID: <200204091750.g39Hoot22844@odiug.zope.com> > > I wasn't serious, but AFAICT /F was serious in his suggestion not to > > discuss my PEPs on c.l.py any more. > > not sure that was what I suggested, really -- my point is > that you cannot treat c.l.py as a representative subset of > the current user base. > > most people are elsewhere, and most users won't see the > PEP at all, no matter where you post it. But what am I to do? c.l.py may not be representative, but neither is python-dev, and I'd like to get feedback from as many folks as possible. Maybe it's just a matter of wording -- make it clear that the feedback won't necessarily affect the PEP unless I like what's being said. But that will still cause certain folks to bitch. Sigh. --Guido van Rossum (home page: http://www.python.org/~guido/) From hughett@mercur.uphs.upenn.edu Tue Apr 9 18:52:34 2002 From: hughett@mercur.uphs.upenn.edu (Paul Hughett) Date: Tue, 9 Apr 2002 13:52:34 -0400 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: <15538.60634.808936.3925@grendel.zope.com> (fdrake@acm.org) References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <02040821105703.14931@arthur> <200204082000.g38K0O914299@pcp742651pcs.reston01.va.comcast.net> <02040822232006.14931@arthur> <200204090314.g393E6E17355@pcp742651pcs.reston01.va.comcast.net> <15538.60634.808936.3925@grendel.zope.com> Message-ID: <200204091752.g39HqYO16556@mercur.uphs.upenn.edu> Fred Drake wrote: > I think it would be a good idea to have a way to determine from a > checkin comment whether it was part of a bugfix (w/ issue # if there > is one), feature (w/ issue #, again), or simple code cleanliness > (whitespace cleanup, etc.). Something that can be checked by tools > would be nice, especially for release managers. The convention that I've seen and adopted for my own use has the first word of the checkin comment be one of the following, including the colon: ENH: Enhancement BUG: Bug fix DOC: Change to comments or whitespace PERF: Performance improvement w/o feature changes Paul Hughett From guido@python.org Tue Apr 9 18:53:02 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 13:53:02 -0400 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: Your message of "Tue, 09 Apr 2002 13:29:09 EDT." <20020409172909.GA4657@panix.com> References: <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <200204091713.g39HDwf22586@odiug.zope.com> <20020409172909.GA4657@panix.com> Message-ID: <200204091753.g39Hr2t22864@odiug.zope.com> > Seems to me that you're being unreasonably, er, boolean in your > thinking here. Yes, as a way to help move the discussion forward. :-) > From my POV, the fact that the BDFL follows the same PEP process > as everyone else is a powerful point in favor of stability. Even if in practice it causes the peasants to show up at the gates armed with pitchforks? > There may not be a way to do it, but I'd sure like to try some > things to lessen the difficulty of using c.l.py. I'd be happy to run my PEPs by the Aahz PR machine. You might want to remind me of that next time I float a PEP in python-dev, i.e. well before it hits c.l.py. --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas.heller@ion-tof.com Tue Apr 9 18:59:43 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Tue, 9 Apr 2002 19:59:43 +0200 Subject: [Python-Dev] New type objects and PyObject_New Message-ID: <0af201c1dff0$54084de0$e000a8c0@thomasnotebook> It seems possible to create new type objects still with PyObject_New() instead of calling the type object. Is this a bug? Thomas From mal@lemburg.com Tue Apr 9 19:04:10 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 09 Apr 2002 20:04:10 +0200 Subject: [Python-Dev] Re: PEP Politics References: <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <200204091713.g39HDwf22586@odiug.zope.com> Message-ID: <3CB32D1A.67EDAB59@lemburg.com> Guido van Rossum wrote: > > > On Tue, Apr 09, 2002 at 12:28:25PM -0400, Guido van Rossum wrote: > > >Maybe I should stop posting PEPs to c.l.py, and tell them that if they > > >want a say in the future of the language, they should subscribe to > > >python-dev? > > > > NOOOOOO!!!!! I don't want to have to start using a killfile on > > python-dev traffic to eliminate endless windy threads that never go > > anywhere. > > > > --amk > > > > PS: NOOOOOO!!!!! > > I wasn't serious, but AFAICT /F was serious in his suggestion not to > discuss my PEPs on c.l.py any more. I think the consequence of that > would be that my PEPs only get discussed on python-dev. Then I see > two possibilities: simply ignore the c.l.py crowd, or inviting them. > I see serious downsides to each. So that means I'll have to face > c.l.py every time I have a PEP, or stop writing PEPs. The latter > sounds the most attractive -- I'll just check things in after a brief > discussion on python-dev. The latter pretty much defeats the whole purpose of PEPs. I wouldn't want to get c.l.p style discussions on python-dev about PEPs, though. Why not setup a special PEP discussion list and then invite people from c.l.p over to it for additional insights ? Regarding the subject line: don't know if it's just me, but I would like to see some of the conservative development style we had established a few years ago return in Python's development process. Some of the recent developments left me under the impression of the need to rush changes with no apparent reason (for rushing them). -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From hughett@mercur.uphs.upenn.edu Tue Apr 9 19:05:07 2002 From: hughett@mercur.uphs.upenn.edu (Paul Hughett) Date: Tue, 9 Apr 2002 14:05:07 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15539.4673.910115.885056@12-248-41-177.client.attbi.com> (message from Skip Montanaro on Tue, 9 Apr 2002 11:09:37 -0500) References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15538.194.350128.498326@12-248-41-177.client.attbi.com> <200204082310.g38NAUD14877@mercur.uphs.upenn.edu> <200204090019.g390J3C15946@pcp742651pcs.reston01.va.comcast.net> <200204091420.g39EKPF16144@mercur.uphs.upenn.edu> <15539.4673.910115.885056@12-248-41-177.client.attbi.com> Message-ID: <200204091805.g39I57g16599@mercur.uphs.upenn.edu> Skip wrote: > Paul> Now I have to confess that, while I've downloaded the > Paul> 2.0 source package, I've been too busy actually using Python to > Paul> open it up and look at the API documentation. > You mean 2.1 (2.1.3 was just released) or 2.2 (2.2.1 is about to be > released), right? Alas, I really do mean 2.0; this has been on my nice-to-do list for more than a year now. I haven't needed any of the new features yet, so there hasn't been much incentive to look for more work. Paul Hughett From skip@pobox.com Tue Apr 9 19:15:47 2002 From: skip@pobox.com (Skip Montanaro) Date: Tue, 9 Apr 2002 13:15:47 -0500 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: <3CB31C6B.5A2FE5D@lemburg.com> References: <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <3CB31C6B.5A2FE5D@lemburg.com> Message-ID: <15539.12243.704667.370457@12-248-41-177.client.attbi.com> mal> I'd suggest to either leave the discussions on c.l.p ... Hey, there's an idea! Let's only post the PEP announcements to comp.lang.perl. That should solve the nasty problem of negative feedback from comp.lang.python... ;-) Skip From skip@pobox.com Tue Apr 9 19:23:36 2002 From: skip@pobox.com (Skip Montanaro) Date: Tue, 9 Apr 2002 13:23:36 -0500 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: <200204091750.g39Hoot22844@odiug.zope.com> References: <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <200204091713.g39HDwf22586@odiug.zope.com> <026601c1dfed$884808f0$ced241d5@hagrid> <200204091750.g39Hoot22844@odiug.zope.com> Message-ID: <15539.12712.625353.978747@12-248-41-177.client.attbi.com> >> most people are elsewhere, and most users won't see the >> PEP at all, no matter where you post it. Guido> But what am I to do? I propose: 1. Do most/all of the serious discussion on python-dev. 2. When http://www.python.org/peps/ changes, post a simple announcement just to c.l.py.announce that such-and-such a PEP was updated or is new. If people are that interested in tracking PEPs, there is the above URL after all. They can register with one of those URL notifier services if they are that intent. Since python.org doesn't host ads, there should be no problem with spurious notifications. I sure a little watcher script for #2 wouldn't be hard to write. Skip From barry@zope.com Tue Apr 9 19:23:05 2002 From: barry@zope.com (Barry A. Warsaw) Date: Tue, 9 Apr 2002 14:23:05 -0400 Subject: [Python-Dev] Re: PEP Politics References: <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <200204091713.g39HDwf22586@odiug.zope.com> <3CB32D1A.67EDAB59@lemburg.com> Message-ID: <15539.12681.269383.530632@anthem.wooz.org> >>>>> "MAL" == M writes: MAL> I wouldn't want to get c.l.p style discussions on python-dev MAL> about PEPs, though. Why not setup a special PEP discussion MAL> list and then invite people from c.l.p over to it for MAL> additional insights ? I thought we rejected this a week or so ago. :) But if that's what people want, it's easy to do. -Barry From aahz@pythoncraft.com Tue Apr 9 19:23:45 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 9 Apr 2002 14:23:45 -0400 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: <3CB32D1A.67EDAB59@lemburg.com> References: <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <200204091713.g39HDwf22586@odiug.zope.com> <3CB32D1A.67EDAB59@lemburg.com> Message-ID: <20020409182344.GB15873@panix.com> On Tue, Apr 09, 2002, M.-A. Lemburg wrote: > > I wouldn't want to get c.l.p style discussions on python-dev about > PEPs, though. Why not setup a special PEP discussion list and then > invite people from c.l.p over to it for additional insights ? > > Regarding the subject line: don't know if it's just me, but I > would like to see some of the conservative development style we > had established a few years ago return in Python's development > process. Some of the recent developments left me under the impression > of the need to rush changes with no apparent reason (for rushing > them). Or, to put it another way, perhaps before development work for each release starts (aside from bug fixes), we draw up a list of the feature goals for that release and project a target date for finishing those goals, but the goals get the focus rather than the date. Changing goals after development starts would require official BDFL prounouncement. If a great feature comes up after development starts, too bad -- the next development cycle will usually be less than nine months away. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From aahz@pythoncraft.com Tue Apr 9 19:25:33 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 9 Apr 2002 14:25:33 -0400 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: <200204091753.g39Hr2t22864@odiug.zope.com> References: <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <200204091713.g39HDwf22586@odiug.zope.com> <20020409172909.GA4657@panix.com> <200204091753.g39Hr2t22864@odiug.zope.com> Message-ID: <20020409182533.GC15873@panix.com> On Tue, Apr 09, 2002, Guido van Rossum wrote: > Aahz: >> >> From my POV, the fact that the BDFL follows the same PEP process >> as everyone else is a powerful point in favor of stability. > > Even if in practice it causes the peasants to show up at the gates > armed with pitchforks? Perhaps, but let's see if we can dissuade the peasants first. >> There may not be a way to do it, but I'd sure like to try some >> things to lessen the difficulty of using c.l.py. > > I'd be happy to run my PEPs by the Aahz PR machine. You might want to > remind me of that next time I float a PEP in python-dev, i.e. well > before it hits c.l.py. Added to my Todo list. ;-0) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From aahz@pythoncraft.com Tue Apr 9 19:19:23 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 9 Apr 2002 14:19:23 -0400 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: <20020409144740.A4422@ibook.distro.conectiva> References: <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <200204091713.g39HDwf22586@odiug.zope.com> <20020409144740.A4422@ibook.distro.conectiva> Message-ID: <20020409181923.GA15873@panix.com> On Tue, Apr 09, 2002, Gustavo Niemeyer wrote: > > > I see serious downsides to each. So that means I'll have to face > > c.l.py every time I have a PEP, or stop writing PEPs. The latter > > sounds the most attractive -- I'll just check things in after a brief > > discussion on python-dev. > > There's another alternative: you could just ask for someone else to > *publish* your PEPs. Yup, that's the next step IMO, if editing the PEPs doesn't work. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From skip@pobox.com Tue Apr 9 19:28:11 2002 From: skip@pobox.com (Skip Montanaro) Date: Tue, 9 Apr 2002 13:28:11 -0500 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: <3CB32D1A.67EDAB59@lemburg.com> References: <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <200204091713.g39HDwf22586@odiug.zope.com> <3CB32D1A.67EDAB59@lemburg.com> Message-ID: <15539.12987.180998.23231@12-248-41-177.client.attbi.com> mal> Regarding the subject line: don't know if it's just me, but I would mal> like to see some of the conservative development style we had mal> established a few years ago return in Python's development process. mal> Some of the recent developments left me under the impression of the mal> need to rush changes with no apparent reason (for rushing them). I think this is a side-effect of having a largely time-based instead of goal-based release process. With a goal-based release process you can effectively say: We will sell no wine before its time. - Orson Welles, in a Paul Masson ad from my childhood Skip From martin@v.loewis.de Tue Apr 9 19:08:26 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 09 Apr 2002 20:08:26 +0200 Subject: [Python-Dev] 2.3 goals In-Reply-To: <200204091723.g39HN8M22644@odiug.zope.com> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <2mr8lpi4mr.fsf@starship.python.net> <15538.65526.586434.980032@12-248-41-177.client.attbi.com> <20020409150657.GA8019@crystal.mems-exchange.org> <20020409163911.GB8520@crystal.mems-exchange.org> <200204091723.g39HN8M22644@odiug.zope.com> Message-ID: Guido van Rossum writes: > > Oh, and another possible 2.3 thing: is it time to drop the PCRE code? > > Yes please, and anything that uses it (like pre.py). I guess sre.py > must remain as mostly an alias for re.py. I would recommend to stop building it; shipping it might still be useful. Likewise for regex? Regards, Martin From pedroni@inf.ethz.ch Tue Apr 9 19:32:08 2002 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Tue, 9 Apr 2002 20:32:08 +0200 Subject: [Python-Dev] Re: PEP Politics References: <15537.17278.333109.580542@anthem.wooz.org> <200204081344.g38DiQY11769@pcp742651pcs.reston01.va.comcast.net> <15537.42265.890383.842760@12-248-41-177.client.attbi.com> <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <200204091713.g39HDwf22586@odiug.zope.com> <026601c1dfed$884808f0$ced241d5@hagrid> <200204091750.g39Hoot22844@odiug.zope.com> Message-ID: <018201c1dff4$db4aa100$6d94fea9@newmexico> From: Guido van Rossum > But what am I to do? c.l.py may not be representative, but neither is > python-dev, and I'd like to get feedback from as many folks as > possible. Maybe it's just a matter of wording -- make it clear that > the feedback won't necessarily affect the PEP unless I like what's > being said. But that will still cause certain folks to bitch. Sigh. > I have the impression (at least for 285) that you got enough feeback on the technical points from python-dev. For the rest the important aspects are: - breakage you typically know what a change will breaks, OTOH you might not know how much code will be concretely affected, for that you could use the experimental track to get feedback from people, admitting that they will try things out and report honestly. But my very personal impression is that you prefer a bit asymptotic language purity over avoiding breakage, so you should expect some over-violent reactions. - newbie friendliness and teaching: I'm happy I am not the BDFL of a self-declared newbie-friendly language . regards. From jeremy@zope.com Tue Apr 9 19:32:37 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Tue, 9 Apr 2002 14:32:37 -0400 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: <20020409182344.GB15873@panix.com> References: <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <200204091713.g39HDwf22586@odiug.zope.com> <3CB32D1A.67EDAB59@lemburg.com> <20020409182344.GB15873@panix.com> Message-ID: <15539.13253.440232.854962@slothrop.zope.com> >>>>> "A" == aahz writes: A> On Tue, Apr 09, 2002, M.-A. Lemburg wrote: >> >> Regarding the subject line: don't know if it's just me, but I >> would like to see some of the conservative development style we >> had established a few years ago return in Python's development >> process. Some of the recent developments left me under the >> impression of the need to rush changes with no apparent reason >> (for rushing them). A> Or, to put it another way, perhaps before development work for A> each release starts (aside from bug fixes), we draw up a list of A> the feature goals for that release and project a target date for A> finishing those goals, but the goals get the focus rather than A> the date. Changing goals after development starts would require A> official BDFL prounouncement. That's pretty much what we do now, isn't it? We consider both a set of functional goals and a release schedule. We don't want to make these decisions in isolation. If there are N features in the release, and all but 1 can be finished in six months, we don't want to hold them all up for one feature that takes two years. So we say the next release will have N-1 features, and the other feature will go in a future release. A> If a great feature comes up after development starts, too bad -- A> the next development cycle will usually be less than nine months A> away. The only person sneaking in new features is Guido <0.2 wink>. Jeremy From fdrake@acm.org Tue Apr 9 19:50:32 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 9 Apr 2002 14:50:32 -0400 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: <15539.13253.440232.854962@slothrop.zope.com> References: <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <200204091713.g39HDwf22586@odiug.zope.com> <3CB32D1A.67EDAB59@lemburg.com> <20020409182344.GB15873@panix.com> <15539.13253.440232.854962@slothrop.zope.com> Message-ID: <15539.14328.615493.938152@grendel.zope.com> Jeremy Hylton writes: > We consider both a set of functional goals and a release schedule. We > don't want to make these decisions in isolation. If there are N That's right. But we definately determine the release schedule without any regard for what else is on our schedule, and we don't make changes to the schedule based on what gets added to our schedule before we're done. Or that's been my perception at least. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From jeremy@zope.com Tue Apr 9 20:03:19 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Tue, 9 Apr 2002 15:03:19 -0400 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: <15539.14328.615493.938152@grendel.zope.com> References: <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <200204091713.g39HDwf22586@odiug.zope.com> <3CB32D1A.67EDAB59@lemburg.com> <20020409182344.GB15873@panix.com> <15539.13253.440232.854962@slothrop.zope.com> <15539.14328.615493.938152@grendel.zope.com> Message-ID: <15539.15095.838844.308712@slothrop.zope.com> >>>>> "FLD" == Fred L Drake, writes: FLD> Jeremy Hylton writes: >> We consider both a set of functional goals and a release >> schedule. We don't want to make these decisions in isolation. >> If there are N FLD> That's right. But we definately determine the release schedule FLD> without any regard for what else is on our schedule, and we FLD> don't make changes to the schedule based on what gets added to FLD> our schedule before we're done. FLD> Or that's been my perception at least. We chose a release schedule a few weeks or months after the previous release, and it always slides. So I think we do adjust the schedule to some extent. Did we add an extra alpha or beta release of the type-class unification? On the other hand, we tend to resist too much slippage after the schedule has been set. There was definitely a lot of pressure to get the __future__ stuff for nested_scopes into 2.1 without causing the schedule to be delayed :-O. Jeremy From skip@pobox.com Tue Apr 9 20:10:05 2002 From: skip@pobox.com (Skip Montanaro) Date: Tue, 9 Apr 2002 14:10:05 -0500 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: <15539.13253.440232.854962@slothrop.zope.com> References: <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <200204091713.g39HDwf22586@odiug.zope.com> <3CB32D1A.67EDAB59@lemburg.com> <20020409182344.GB15873@panix.com> <15539.13253.440232.854962@slothrop.zope.com> Message-ID: <15539.15501.505596.95324@12-248-41-177.client.attbi.com> A> Or, to put it another way, perhaps before development work for each A> release starts (aside from bug fixes), we draw up a list of the A> feature goals for that release ... Jeremy> That's pretty much what we do now, isn't it? Hmm... I must have missed that PEP. There is a section in the 2.3 Release Schedule entitled "Planned features for 2.3". Is that what you're referring to? I would toss the time information altogether and rename the PEP "Planned Features for 2.x". I just haven't seen enough justification for identifying specific release dates, particularly early on, before new code has been added to the repository and had some testing. We may well be talking more about perception than reality, but it seems to me that placing fairly artificial time limits on releases can only serve to reinforce the perception of instability. Skip From DavidA@ActiveState.com Tue Apr 9 20:08:44 2002 From: DavidA@ActiveState.com (David Ascher) Date: Tue, 09 Apr 2002 12:08:44 -0700 Subject: [Python-Dev] Re: PEP Politics References: <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <200204091713.g39HDwf22586@odiug.zope.com> <3CB32D1A.67EDAB59@lemburg.com> <20020409182344.GB15873@panix.com> <15539.13253.440232.854962@slothrop.zope.com> <15539.14328.615493.938152@grendel.zope.com> Message-ID: <3CB33C3C.33A208DB@activestate.com> "Fred L. Drake, Jr." wrote: > That's right. But we definately determine the release schedule > without any regard for what else is on our schedule, and we don't make > changes to the schedule based on what gets added to our schedule > before we're done. That sounds wrong. However, I think there's still value in schedules. Without schedules things just don't get done, in my experience. --da From Jack.Jansen@oratrix.com Tue Apr 9 20:23:59 2002 From: Jack.Jansen@oratrix.com (Jack Jansen) Date: Tue, 9 Apr 2002 21:23:59 +0200 Subject: [Python-Dev] 2.2.1 release mechanics In-Reply-To: <200204082152.g38LqeS15350@pcp742651pcs.reston01.va.comcast.net> Message-ID: <5773CE56-4BEF-11D6-BA9A-003065517236@oratrix.com> On maandag, april 8, 2002, at 11:52 , Guido van Rossum wrote: >> + I believe we're no longer uploading anything to SF, although I'm >> not sure Michael and Anthony both believe that. > > AFAIK they were cc'ed on the msgs where that was decided. Uhm... Are we talking about uploading releases to sourceforge here? Do I understand that we're not doing this anymore? -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From barry@zope.com Tue Apr 9 20:28:19 2002 From: barry@zope.com (Barry A. Warsaw) Date: Tue, 9 Apr 2002 15:28:19 -0400 Subject: [Python-Dev] Regression in unicodestr.encode()? Message-ID: <15539.16595.651881.216153@yyz.zope.com> I'm porting over the latest email package to Python 2.3cvs, and I've had one of my tests fail. I've narrowed it down to the following test case: a = u'\u6b63\u78ba\u306b\u8a00\u3046\u3068\u7ffb\u8a33\u306f\u3055\u308c\u3066\u3044\u307e\u305b\u3093\u3002\u4e00\u90e8\u306f\u30c9\u30a4\u30c4\u8a9e\u3067\u3059\u304c\u3001\u3042\u3068\u306f\u3067\u305f\u3089\u3081\u3067\u3059\u3002\u5b9f\u969b\u306b\u306f\u300cWenn ist das Nunstuck git und' print repr(a.encode('utf-8', 'replace')) In Python 2.2.1 I get '\xe6\xad\xa3\xe7\xa2\xba\xe3\x81\xab\xe8\xa8\x80\xe3\x81\x86\xe3\x81\xa8\xe7\xbf\xbb\xe8\xa8\xb3\xe3\x81\xaf\xe3\x81\x95\xe3\x82\x8c\xe3\x81\xa6\xe3\x81\x84\xe3\x81\xbe\xe3\x81\x9b\xe3\x82\x93\xe3\x80\x82\xe4\xb8\x80\xe9\x83\xa8\xe3\x81\xaf\xe3\x83\x89\xe3\x82\xa4\xe3\x83\x84\xe8\xaa\x9e\xe3\x81\xa7\xe3\x81\x99\xe3\x81\x8c\xe3\x80\x81\xe3\x81\x82\xe3\x81\xa8\xe3\x81\xaf\xe3\x81\xa7\xe3\x81\x9f\xe3\x82\x89\xe3\x82\x81\xe3\x81\xa7\xe3\x81\x99\xe3\x80\x82\xe5\xae\x9f\xe9\x9a\x9b\xe3\x81\xab\xe3\x81\xaf\xe3\x80\x8cWenn ist das Nunstuck git und' but in Python 2.3 cvs I get '\xe6\xad\xa3\xe7\xa2\xba\xe3\x81\xab\xe8\xa8\x80\xe3\x81\x86\xe3\x81\xa8\xe7\xbf\xbb\xe8\xa8\xb3\xe3\x81\xaf\xe3\x81\x95\xe3\x82\x8c\xe3\x81\xa6\xe3\x81\x84\xe3\x81\xbe\xe3\x81\x9b\xe3\x82\x93\xe3\x80\x82\xe4\xb8\x80\xe9\x83\xa8\xe3\x81\xaf\xe3\x83\x89\xe3\x82\xa4\xe3\x83\x84\xe8\xaa\x9e\xe3\x81\xa7\xe3\x81\x99\xe3\x81\x8c\xe3\x80\x81\xe3\x81\x82\xe3\x81\xa8\xe3\x81\xaf\xe3\x81\xa7\xe3\x81\x9f\xe3\x82\x89\xe3\x82\x81\xe3\x81\xa7\xe3\x81\x99\xe3\x80\x82\xe5\xae\x9f\xe9\x9a\x9b\xe3\x81\xab\xe3\x81\xaf\xe3\x80\x8cWenn ist das Nunstuck git u\x00\x00' Note that the last two characters, which should be `n' and `d' are now NULs. My very limited Tim-enlightened understanding is that encoding a string to UTF-8 should never produce a string with NULs. Is the utf-8 encoder in cvs broken? -Barry From tim@zope.com Tue Apr 9 20:36:44 2002 From: tim@zope.com (Tim Peters) Date: Tue, 9 Apr 2002 15:36:44 -0400 Subject: [Python-Dev] 2.2.1 release mechanics In-Reply-To: <5773CE56-4BEF-11D6-BA9A-003065517236@oratrix.com> Message-ID: [Jack Jansen] > Uhm... Are we talking about uploading releases to sourceforge > here? Right. > Do I understand that we're not doing this anymore? It's true that we're not uploading releases to SourceForge anymore. Whether you understand that is between you and your God . From tim@zope.com Tue Apr 9 20:52:34 2002 From: tim@zope.com (Tim Peters) Date: Tue, 9 Apr 2002 15:52:34 -0400 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: <200204091753.g39Hr2t22864@odiug.zope.com> Message-ID: [Aahz] >> From my POV, the fact that the BDFL follows the same PEP process >> as everyone else is a powerful point in favor of stability. [Guido] > Even if in practice it causes the peasants to show up at the gates > armed with pitchforks? Probably, yes: nobody pitching a PEP on c.l.py gets off easy. By the time someone has posted a few revisions of any PEP, they've been through fire. You just get more of everything (both pro and con). Keep that in mind next time you reject a PEP -- the author went through a minor suburb of hell to carry it that far. From martin@v.loewis.de Tue Apr 9 20:39:09 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 09 Apr 2002 21:39:09 +0200 Subject: [Python-Dev] New type objects and PyObject_New In-Reply-To: <0af201c1dff0$54084de0$e000a8c0@thomasnotebook> References: <0af201c1dff0$54084de0$e000a8c0@thomasnotebook> Message-ID: "Thomas Heller" writes: > It seems possible to create new type objects still with > PyObject_New() instead of calling the type object. > Is this a bug? Not sure what you are asking: I assume you are *not* asking about new type objects (as created by typeobject.c:type_new), but instead about instance of newstyle classes (i.e. instance of type objects). Why would you then think that using PyObject_New is a bad thing? Regards, Martin From tim@zope.com Tue Apr 9 21:00:29 2002 From: tim@zope.com (Tim Peters) Date: Tue, 9 Apr 2002 16:00:29 -0400 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: <3CB33C3C.33A208DB@activestate.com> Message-ID: [David Ascher] > ... > However, I think there's still value in schedules. Without schedules > things just don't get done, in my experience. Or at least deadlines. Note that this ties in with frequent releases: if we said we won't release something "until it's ready", and the time between releases slobbers out to over a year again, then people will fight like hell to get "just one more" change in -- over and over and over again. Almost nothing is ever really finished. Frequent releases make it much easier to sell "too bad, you're too late for this release, but you'll get another chance in 6 months". From thomas.heller@ion-tof.com Tue Apr 9 21:06:22 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Tue, 9 Apr 2002 22:06:22 +0200 Subject: [Python-Dev] New type objects and PyObject_New References: <0af201c1dff0$54084de0$e000a8c0@thomasnotebook> Message-ID: <0cd501c1e002$0551fc20$e000a8c0@thomasnotebook> From: "Martin v. Loewis" > "Thomas Heller" writes: > > > It seems possible to create new type objects still with > > PyObject_New() instead of calling the type object. > > Is this a bug? > > Not sure what you are asking: I assume you are *not* asking about new > type objects (as created by typeobject.c:type_new), but instead about > instance of newstyle classes (i.e. instance of type objects). Correct. > > Why would you then think that using PyObject_New is a bad thing? IIUC, PyObject_New doesn't call the tp_new, tp_alloc, and tp_init slots. So, the new object probably will be in an invalid state, may be allocated by an allocator not compatible with the tp_dealloc slot? > > Regards, > Martin > > Thanks, Thomas From tim.one@comcast.net Tue Apr 9 21:12:53 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 09 Apr 2002 16:12:53 -0400 Subject: [Python-Dev] 2.3 goals In-Reply-To: <20020409163911.GB8520@crystal.mems-exchange.org> Message-ID: [Andrew Kuchling] > Oh, and another possible 2.3 thing: is it time to drop the PCRE code? Whatever /F says. Over and over we've gotten reports that some naively written regexps that work OK (albeit slowly) under pre "blow the stack" under sre. The suggested workaround has been to "import pre as re". So we break working code if the pre engine goes away before /F thinks sre is as forgiving in these nasty cases. From martin@v.loewis.de Tue Apr 9 21:19:04 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 09 Apr 2002 22:19:04 +0200 Subject: [Python-Dev] New type objects and PyObject_New In-Reply-To: <0cd501c1e002$0551fc20$e000a8c0@thomasnotebook> References: <0af201c1dff0$54084de0$e000a8c0@thomasnotebook> <0cd501c1e002$0551fc20$e000a8c0@thomasnotebook> Message-ID: "Thomas Heller" writes: > IIUC, PyObject_New doesn't call the tp_new, tp_alloc, and tp_init slots. > So, the new object probably will be in an invalid state, may be > allocated by an allocator not compatible with the tp_dealloc slot? I see. However, how do you implement tp_new if PyObject_New went away? Regards, Martin From aahz@pythoncraft.com Tue Apr 9 22:03:17 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 9 Apr 2002 17:03:17 -0400 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: References: <3CB2E244.6010102@strakt.com> Message-ID: <20020409210317.GA16531@panix.com> On Tue, Apr 09, 2002, Alex Martelli wrote: > > I suspect the situation may change by distribution. Mandrake, if I know > them well (and I'm a satisfied customer for over a year, supporting > member of the Mandrake club, etc), WILL keep on top of the bleeding > edge, releasing a stable-python and an experimental-python package, > or something like that -- they know their customer base includes a lot > of rabid neophiles, as well as people who only want stability. Debian, > though, I wonder -- I doubt they'll put in their "Stable" distribution > anything not vouchsafed for by Methushelah in person:-). As for > RedHat, no problem -- they'll keep shipping 1.5.2 anyway...!-) Right, but I think that for all of these, there needs to be an actual "release", not just a CVS snapshot they grab themselves. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From aahz@pythoncraft.com Tue Apr 9 22:09:43 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 9 Apr 2002 17:09:43 -0400 Subject: [Python-Dev] Unittests In-Reply-To: <3CB3093C.B7A22727@metaslash.com> References: <3CB3093C.B7A22727@metaslash.com> Message-ID: <20020409210943.GB16531@panix.com> On Tue, Apr 09, 2002, Neal Norwitz wrote: > > whrandom.py This is a deprecated module, no? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From pobrien@orbtech.com Tue Apr 9 22:22:13 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Tue, 9 Apr 2002 16:22:13 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <3CB3093C.B7A22727@metaslash.com> Message-ID: [Neal Norwitz] > > "Patrick K. O'Brien" wrote: > > > What I think is needed is a list of what is missing > > The list is quite large. Skip the rest of this messge for those > who are uninterested. The list contains both python modules > as well as C modules. Please let me know about mistakes. [big list of modules missing unit tests] Thanks for the list. I'm curious now. Where did this list come from? Is it actively maintained? If we start adding the missing unit tests is there a way to maintain this list? --- Patrick K. O'Brien Orbtech From neal@metaslash.com Tue Apr 9 22:24:20 2002 From: neal@metaslash.com (Neal Norwitz) Date: Tue, 09 Apr 2002 17:24:20 -0400 Subject: [Python-Dev] Re: Unittests References: <3CB3093C.B7A22727@metaslash.com> <20020409210943.GB16531@panix.com> Message-ID: <3CB35C04.F6EC6AD3@metaslash.com> Aahz wrote: > > On Tue, Apr 09, 2002, Neal Norwitz wrote: > > > > whrandom.py > > This is a deprecated module, no? It appears only whrandom.randint() is deprecated. There is also a random module. The rand module is in lib-old. Neal From pobrien@orbtech.com Tue Apr 9 22:32:57 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Tue, 9 Apr 2002 16:32:57 -0500 Subject: [Python-Dev] Unittests In-Reply-To: <20020409210943.GB16531@panix.com> Message-ID: [Aahz] > > > > whrandom.py > > This is a deprecated module, no? Except Zope still uses it quite a bit for some reason. --- Patrick K. O'Brien Orbtech From neal@metaslash.com Tue Apr 9 22:30:01 2002 From: neal@metaslash.com (Neal Norwitz) Date: Tue, 09 Apr 2002 17:30:01 -0400 Subject: [Python-Dev] Re: Stability and change References: Message-ID: <3CB35D59.E4AFFB35@metaslash.com> "Patrick K. O'Brien" wrote: > > [Neal Norwitz] > > > > "Patrick K. O'Brien" wrote: > > > > > What I think is needed is a list of what is missing > > > > The list is quite large. Skip the rest of this messge for those > > who are uninterested. The list contains both python modules > > as well as C modules. Please let me know about mistakes. > > [big list of modules missing unit tests] > > Thanks for the list. I'm curious now. Where did this list come from? Is it > actively maintained? If we start adding the missing unit tests is there a > way to maintain this list? I pulled it from where the sun don't shine. :-) I did a bunch of ls/grep/echo/... commands. Then post processed manually. It's not maintained anywhere. It would be great if there was a place to maintain/update it. Suggestions? I have another script which builds python with test-coverage and will tell you each line/function that doesn't get executed in case you get really bored. :-) There's only about 200 functions in Objects/*.c. Neal From Jack.Jansen@oratrix.com Tue Apr 9 22:39:42 2002 From: Jack.Jansen@oratrix.com (Jack Jansen) Date: Tue, 9 Apr 2002 23:39:42 +0200 Subject: [Python-Dev] 2.2.1 release mechanics In-Reply-To: <2m7knspj43.fsf@starship.python.net> Message-ID: <4D430248-4C02-11D6-BA9A-003065517236@oratrix.com> On zondag, maart 31, 2002, at 10:39 , Michael Hudson wrote: > April 10 ~1200 GMT: release ends > > By now, F, T & J have done their bits, uploaded > files to creosote and sf (or pointed me to where I > can get them), etc. Michael, my stuff is done. I've announced it on the pythonmac-sig and given people there 12 hours to find any showstopper bugs, so unless I ask you not to go ahead within twelve hours it's all yours (I understand that we're not doing sf releases anymore, and I don't have access to creosote). The files are in http://www.cwi.nl/ftp/jack/python/mac, you want MacPython221full (both .hqx and .bin) and MacPython221src (both .hqx and .sit). You may also want MacPython221MacOS81patch (.hqx and .sit) which makes this release run on really old MacOS versions. If you are going to put a blurb in your message announcing 2.2.1 there are two fixes in it that are very important to Mac users: - This release finally works on Mac OS X multiprocessors - The IDE is now fully functional on Mac OS X. -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From pinard@iro.umontreal.ca Tue Apr 9 23:15:39 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 09 Apr 2002 18:15:39 -0400 Subject: [Python-Dev] Re: Regression in unicodestr.encode()? In-Reply-To: <15539.16595.651881.216153@yyz.zope.com> References: <15539.16595.651881.216153@yyz.zope.com> Message-ID: [Barry A. Warsaw] > My very limited Tim-enlightened understanding is that encoding a string > to UTF-8 should never produce a string with NULs. Besides the fact that NULs encode to themselves. In fact, 0-127 encode to themselves, and are never produced otherwise. Also, 254 and 255 are never produced, I heard that some use these to escape out within an UTF-8 string. -- François Pinard http://www.iro.umontreal.ca/~pinard From David Abrahams" Message-ID: <050701c1e015$26bf97b0$6601a8c0@boostconsulting.com> ----- Original Message ----- From: "Thomas Heller" > It seems possible to create new type objects still with > PyObject_New() instead of calling the type object. > Is this a bug? I sure hope not! I need that functionality! -Dave From bsder@allcaps.org Tue Apr 9 23:49:10 2002 From: bsder@allcaps.org (Andrew P. Lentvorski) Date: Tue, 9 Apr 2002 15:49:10 -0700 (PDT) Subject: [Python-Dev] Unittests In-Reply-To: <20020409210943.GB16531@panix.com> Message-ID: <20020409153410.Y30037-100000@mail.allcaps.org> On Tue, 9 Apr 2002, Aahz wrote: > On Tue, Apr 09, 2002, Neal Norwitz wrote: > > > > whrandom.py > > This is a deprecated module, no? What is the "Official Python(tm)" method for generating a "guaranteed for all time to always be the same if started with the same seed" string of random numbers for use in something like a UnitTest? The Random module does not seem to make any promises about repeatability between versions, releases, operating-systems, etc. The only thing which enforces the repeatability is the fact that the generator is actually the whrandom generator under the covers. Incidentally, a similar question has been asked before and didn't really get good closure on the response, see: http://groups.google.com/groups?hl=en&threadm=MPG.16a822cc6abaa77c98970e%40enews.newsguy.com&rnum=1&prev=/groups%3Fq%3Ddanil%2Bwhrandom%26hl%3Den%26selm%3DMPG.16a822cc6abaa77c98970e%2540enews.newsguy.com%26rnum%3D1 (I hope that URL comes through okay, if not search for "danil" and "whrandom" in your favorite clp search engine) -a From guido@python.org Tue Apr 9 23:54:54 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 18:54:54 -0400 Subject: [Python-Dev] New type objects and PyObject_New In-Reply-To: Your message of "Tue, 09 Apr 2002 17:00:11 CDT." <050701c1e015$26bf97b0$6601a8c0@boostconsulting.com> References: <0af201c1dff0$54084de0$e000a8c0@thomasnotebook> <050701c1e015$26bf97b0$6601a8c0@boostconsulting.com> Message-ID: <200204092254.g39Mss118607@pcp742651pcs.reston01.va.comcast.net> [Thomas Heller] > > It seems possible to create new type objects still with > > PyObject_New() instead of calling the type object. > > Is this a bug? [David Abrahams] > I sure hope not! I need that functionality! Um, PyObject_New() is the low-level allocation function that you should be calling from the type's tp_new, not something you should be calling elsewhere to create an instance of a type. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 9 23:57:27 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 18:57:27 -0400 Subject: [Python-Dev] Re: Regression in unicodestr.encode()? In-Reply-To: Your message of "09 Apr 2002 18:15:39 EDT." References: <15539.16595.651881.216153@yyz.zope.com> Message-ID: <200204092257.g39MvRi18636@pcp742651pcs.reston01.va.comcast.net> > [Barry A. Warsaw] > > My very limited Tim-enlightened understanding is that encoding a string > > to UTF-8 should never produce a string with NULs. [François] > Besides the fact that NULs encode to themselves. In fact, 0-127 encode to > themselves, and are never produced otherwise. Also, 254 and 255 are never > produced, I heard that some use these to escape out within an UTF-8 string. Hm, but isn't there a way to encode a NUL that doesn't produce a NUL? In some variant? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Apr 10 00:00:31 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 19:00:31 -0400 Subject: [Python-Dev] Unittests In-Reply-To: Your message of "Tue, 09 Apr 2002 16:32:57 CDT." References: Message-ID: <200204092300.g39N0Vl18669@pcp742651pcs.reston01.va.comcast.net> > > > whrandom.py > > > > This is a deprecated module, no? Almost -- its *implementation* remains the basis of random.py. > Except Zope still uses it quite a bit for some reason. Shouldn't be a problem since Zope 2.x needs Python 2.1.x anyway, and Zope 3 will surely do whatever it takes to run with Python 2.3. --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@pythoncraft.com Wed Apr 10 00:01:19 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 9 Apr 2002 19:01:19 -0400 Subject: [Python-Dev] Unittests In-Reply-To: <200204092300.g39N0Vl18669@pcp742651pcs.reston01.va.comcast.net> References: <200204092300.g39N0Vl18669@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020409230118.GA10503@panix.com> On Tue, Apr 09, 2002, Guido van Rossum wrote: > >>>> whrandom.py >>> >>> This is a deprecated module, no? > > Almost -- its *implementation* remains the basis of random.py. Really? Looking in random.py for 2.2, I find no mention of whrandom. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From aahz@pythoncraft.com Wed Apr 10 00:03:08 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 9 Apr 2002 19:03:08 -0400 Subject: [Python-Dev] Unittests In-Reply-To: <20020409153410.Y30037-100000@mail.allcaps.org> References: <20020409210943.GB16531@panix.com> <20020409153410.Y30037-100000@mail.allcaps.org> Message-ID: <20020409230308.GB10503@panix.com> On Tue, Apr 09, 2002, Andrew P. Lentvorski wrote: > > What is the "Official Python(tm)" method for generating a "guaranteed for > all time to always be the same if started with the same seed" string of > random numbers for use in something like a UnitTest? There isn't one currently, and I doubt there will ever be one. No reason why you can't swipe random.py for your own purposes, though. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From aahz@pythoncraft.com Wed Apr 10 00:05:54 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 9 Apr 2002 19:05:54 -0400 Subject: [Python-Dev] NEWS consolidation In-Reply-To: <20020409144717.GA7946@crystal.mems-exchange.org> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15538.194.350128.498326@12-248-41-177.client.attbi.com> <200204082310.g38NAUD14877@mercur.uphs.upenn.edu> <200204090019.g390J3C15946@pcp742651pcs.reston01.va.comcast.net> <200204091420.g39EKPF16144@mercur.uphs.upenn.edu> <20020409144717.GA7946@crystal.mems-exchange.org> Message-ID: <20020409230554.GC10503@panix.com> On Tue, Apr 09, 2002, Andrew Kuchling wrote: > > Side note: The NEWS file is split into separate parts for each alpha > and beta release; at some point, should all the entries for 2.2a1, > 2.2a2, 2.2b1, &c., be merged into a single section for 2.2final? > Otherwise finding all the C API changes in 2.2 requires a lot of > scrolling and hunting. +1 -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From guido@python.org Wed Apr 10 00:19:35 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 19:19:35 -0400 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: Your message of "Tue, 09 Apr 2002 14:23:45 EDT." <20020409182344.GB15873@panix.com> References: <200204081433.g38EXvX12148@pcp742651pcs.reston01.va.comcast.net> <20020408220120.GB562@panix.com> <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <200204091713.g39HDwf22586@odiug.zope.com> <3CB32D1A.67EDAB59@lemburg.com> <20020409182344.GB15873@panix.com> Message-ID: <200204092319.g39NJZA18841@pcp742651pcs.reston01.va.comcast.net> > If a great feature comes up after development starts, too bad -- the > next development cycle will usually be less than nine months away. Bah. For most features, 9 months is an eternity compared to the time it takes to code it. About the only exceptions I recall are new-style classes and Unicode; major packages like email or xml also are exceptions, but they are usually developed outside the Python CVS tree first anyway. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Apr 10 00:26:09 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 19:26:09 -0400 Subject: [Python-Dev] Unittests In-Reply-To: Your message of "Tue, 09 Apr 2002 15:49:10 PDT." <20020409153410.Y30037-100000@mail.allcaps.org> References: <20020409153410.Y30037-100000@mail.allcaps.org> Message-ID: <200204092326.g39NQAa18909@pcp742651pcs.reston01.va.comcast.net> > What is the "Official Python(tm)" method for generating a > "guaranteed for all time to always be the same if started with the > same seed" string of random numbers for use in something like a > UnitTest? That's a good use case, although the WH algorithm is so simple that it would probably be easier to isolate yourself from evolution of the library by recoding it in that's really what you want. > The Random module does not seem to make any promises about repeatability > between versions, releases, operating-systems, etc. The only thing which > enforces the repeatability is the fact that the generator is actually the > whrandom generator under the covers. > > Incidentally, a similar question has been asked before and didn't really > get good closure on the response, see: > > http://groups.google.com/groups?hl=en&threadm=MPG.16a822cc6abaa77c98970e%40enews.newsguy.com&rnum=1&prev=/groups%3Fq%3Ddanil%2Bwhrandom%26hl%3Den%26selm%3DMPG.16a822cc6abaa77c98970e%2540enews.newsguy.com%26rnum%3D1 Tim didn't ask me before he responded; I might have suggested that deprecating whrandom was a bit premature given your use case. I can certainly see the use of providing a given random algorithm for those who want stability, while at the same time letting the random module migrate to a stronger algorithm for the benefit for humanity. But then again, Tim might have a good reason why that's a bad idea. I don't know. --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@pythoncraft.com Wed Apr 10 00:23:41 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 9 Apr 2002 19:23:41 -0400 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: <200204092319.g39NJZA18841@pcp742651pcs.reston01.va.comcast.net> References: <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <200204091713.g39HDwf22586@odiug.zope.com> <3CB32D1A.67EDAB59@lemburg.com> <20020409182344.GB15873@panix.com> <200204092319.g39NJZA18841@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020409232341.GA15971@panix.com> On Tue, Apr 09, 2002, Guido van Rossum wrote: > >> If a great feature comes up after development starts, too bad -- the >> next development cycle will usually be less than nine months away. > > Bah. For most features, 9 months is an eternity compared to the time > it takes to code it. About the only exceptions I recall are new-style > classes and Unicode; major packages like email or xml also are > exceptions, but they are usually developed outside the Python CVS tree > first anyway. It's not the time for coding that's the issue, it's the time for testing, documentation, integration, and so on. But even that's beside the point; this suggestion is being brought up in the context of the stability/change thread, and this suggestion would IMO go a long way toward changing perceptions. For example, what's the cost in postponing the bool() change to 2.4? (I'm not talking about adding the builtin the way you just did for 2.2.1 (which I agree with), but changing the type of "not x".) Note that I'm not pushing this suggestion, but I think your "Bah" is overstating the opposition. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From guido@python.org Wed Apr 10 00:29:08 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 19:29:08 -0400 Subject: [Python-Dev] NEWS consolidation In-Reply-To: Your message of "Tue, 09 Apr 2002 19:05:54 EDT." <20020409230554.GC10503@panix.com> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15538.194.350128.498326@12-248-41-177.client.attbi.com> <200204082310.g38NAUD14877@mercur.uphs.upenn.edu> <200204090019.g390J3C15946@pcp742651pcs.reston01.va.comcast.net> <200204091420.g39EKPF16144@mercur.uphs.upenn.edu> <20020409144717.GA7946@crystal.mems-exchange.org> <20020409230554.GC10503@panix.com> Message-ID: <200204092329.g39NT8o18942@pcp742651pcs.reston01.va.comcast.net> > > Side note: The NEWS file is split into separate parts for each alpha > > and beta release; at some point, should all the entries for 2.2a1, > > 2.2a2, 2.2b1, &c., be merged into a single section for 2.2final? > > Otherwise finding all the C API changes in 2.2 requires a lot of > > scrolling and hunting. > > +1 I think that maybe the NEWS file should be turned into a database from which different reports can be generated. The database could be disguised as a text file in some simple format; given the use case XML isn't high on my list of suggestions. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From David Abrahams" <050701c1e015$26bf97b0$6601a8c0@boostconsulting.com> <200204092254.g39Mss118607@pcp742651pcs.reston01.va.comcast.net> Message-ID: <003401c1e01f$202777b0$6801a8c0@boostconsulting.com> Uh, never mind, I'm actually doing this: // Don't call the type directly to do the construction, since // that would require the registration of an appropriate // __init__ function. PyObject* raw_result = m_class_object->tp_alloc(m_class_object, 0); ----- Original Message ----- From: "Guido van Rossum" To: "David Abrahams" Cc: "Thomas Heller" ; Sent: Tuesday, April 09, 2002 5:54 PM Subject: Re: [Python-Dev] New type objects and PyObject_New > [Thomas Heller] > > > It seems possible to create new type objects still with > > > PyObject_New() instead of calling the type object. > > > Is this a bug? > > [David Abrahams] > > I sure hope not! I need that functionality! > > Um, PyObject_New() is the low-level allocation function that you > should be calling from the type's tp_new, not something you should be > calling elsewhere to create an instance of a type. > > --Guido van Rossum (home page: http://www.python.org/~guido/) > From python@rcn.com Wed Apr 10 00:43:37 2002 From: python@rcn.com (Raymond Hettinger) Date: Tue, 9 Apr 2002 19:43:37 -0400 Subject: [Python-Dev] Re: PEP Politics References: <20020409225802.10180.75562.Mailman@mail.python.org> Message-ID: <001e01c1e020$5f6def20$7a41accf@othello> From: "Tim Peters" > nobody pitching a PEP on c.l.py gets off easy. By the time > someone has posted a few revisions of any PEP, they've been through fire. > You just get more of everything (both pro and con). Keep that in mind next > time you reject a PEP -- the author went through a minor suburb of hell to > carry it that far. Truer words were never spoken. Raymond From David Abrahams" <050701c1e015$26bf97b0$6601a8c0@boostconsulting.com> <200204092254.g39Mss118607@pcp742651pcs.reston01.va.comcast.net> <003401c1e01f$202777b0$6801a8c0@boostconsulting.com> Message-ID: <005e01c1e020$76b65280$6801a8c0@boostconsulting.com> ----- Original Message ----- From: "David Abrahams" To: "Guido van Rossum" Cc: "Thomas Heller" ; Sent: Tuesday, April 09, 2002 6:33 PM Subject: Re: [Python-Dev] New type objects and PyObject_New > Uh, never mind, I'm actually doing this: > > // Don't call the type directly to do the construction, since > // that would require the registration of an appropriate > // __init__ function. > PyObject* raw_result = m_class_object->tp_alloc(m_class_object, > 0); And no, that's not an emoticon. From guido@python.org Wed Apr 10 01:05:09 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 20:05:09 -0400 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: Your message of "Tue, 09 Apr 2002 19:23:41 EDT." <20020409232341.GA15971@panix.com> References: <200204082235.g38MZv615653@pcp742651pcs.reston01.va.comcast.net> <20020408224359.GE562@panix.com> <200204090104.g39147a16099@pcp742651pcs.reston01.va.comcast.net> <010a01c1dfe1$c177e3e0$0900a8c0@spiff> <200204091628.g39GSQf22128@odiug.zope.com> <20020409164501.GD8520@crystal.mems-exchange.org> <200204091713.g39HDwf22586@odiug.zope.com> <3CB32D1A.67EDAB59@lemburg.com> <20020409182344.GB15873@panix.com> <200204092319.g39NJZA18841@pcp742651pcs.reston01.va.comcast.net> <20020409232341.GA15971@panix.com> Message-ID: <200204100005.g3A059O19004@pcp742651pcs.reston01.va.comcast.net> [Aahz] > >> If a great feature comes up after development starts, too bad -- the > >> next development cycle will usually be less than nine months away. [me] > > Bah. For most features, 9 months is an eternity compared to the time > > it takes to code it. About the only exceptions I recall are new-style > > classes and Unicode; major packages like email or xml also are > > exceptions, but they are usually developed outside the Python CVS tree > > first anyway. [Aahz] > It's not the time for coding that's the issue, it's the time for > testing, documentation, integration, and so on. But even that's > beside the point; this suggestion is being brought up in the context > of the stability/change thread, and this suggestion would IMO go a > long way toward changing perceptions. It depends a lot on the impact of the feature. If something like list comprehensions came up at the last moment, I would agree to put it off. But something like sys._getframe() shouldn't need to wait 9 months, no matter *how* conservative you want to be. I don't think you can give hard and fast rules here. > For example, what's the cost in postponing the bool() change to 2.4? > (I'm not talking about adding the builtin the way you just did for 2.2.1 > (which I agree with), but changing the type of "not x".) > > Note that I'm not pushing this suggestion, but I think your "Bah" is > overstating the opposition. I disagree with your suggestion to put off bool() to 2.4. I see no technical reason to do so, and I don't think politics should influence release decisions to this extent. --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@pythoncraft.com Wed Apr 10 01:20:11 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 9 Apr 2002 20:20:11 -0400 Subject: [Python-Dev] Developer resources (was Re: Stability and change) In-Reply-To: <200204091721.g39HLWm22627@odiug.zope.com> References: <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> <15538.1515.580133.785503@12-248-41-177.client.attbi.com> <15538.2623.323978.125527@slothrop.zope.com> <20020408223655.GD562@panix.com> <15538.32008.560965.639327@grendel.zope.com> <006f01c1dfe3$8ce6c9f0$8001a8c0@COMPUTER> <15539.6606.780921.399696@grendel.zope.com> <200204091721.g39HLWm22627@odiug.zope.com> Message-ID: <20020410002010.GA25648@panix.com> On Tue, Apr 09, 2002, Guido van Rossum wrote: > > The problem with starship is that there are no criteria to admit new > users. There wasn't any difficulty for me in getting a Starship account a bit more than a year ago. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From jafo@tummy.com Wed Apr 10 01:23:17 2002 From: jafo@tummy.com (Sean Reifschneider) Date: Tue, 9 Apr 2002 18:23:17 -0600 Subject: [Python-Dev] Pre-release 2.2.1 RPMs Message-ID: <20020409182317.W16962@tummy.com> I've built RPMs for Red Hat/KRUD 7.2 and 6.2, and Mandrake 8.2, and have uploaded them to cerosote:~jafo/RPMS. Additionally, the Python 2.2.1 tar file will build into a set of RPMs by doing "rpm -ta Python-2.2.1.tgz"... If any of the development team wants to grab them and check them out, go for it. You have about 6 hours before I call them "good" and don't think about them any more until tomorrow. If you find any problems, please let me know. If you have a platform which you feel isn't adequately represented in the above list, I'd be happy to help get RPMs built for them. Contact me via e-mail. Also, tonight from 8pm to 11pm Mountain time (GMT+6), I will be on irc.community.tummy.com, #hackingsociety doing work on the RPMs portion of the web-site. I'd love to see a Mac/PPC RPM, but I'm not sure the Mac that's likely to be at Hacking Society tonight is really up to date enough. Thanks, Sean -- I think you are blind to the fact that the hand you hold is the hand that holds you down. -- Everclear Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From tim.one@comcast.net Wed Apr 10 01:26:25 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 09 Apr 2002 20:26:25 -0400 Subject: random vs whrandom (was RE: [Python-Dev] Unittests) In-Reply-To: <200204092300.g39N0Vl18669@pcp742651pcs.reston01.va.comcast.net> Message-ID: >>>> whrandom.py >>> This is a deprecated module, no? As its docs say, "Deprecated since release 2.1. Use random instead." Somebody toss in DeprecationWarnings or it will never go away. >> Almost -- its *implementation* remains the basis of random.py. No, whrandom.py has been dead weight since 2.1. Delete it from your hard drive, and random.py won't even notice. > Except Zope still uses it quite a bit for some reason. A lot of people still seem to use it for some reason. It's baffling. When I decoupled whrandom from random for 2.1, I added a bunch of "cool stuff" exclusively to the latter to try to induce people to switch. There must be a popular book or guide out there using whrandom, as newcomers to the Tutor list spontaneously post code importing whrandom. From pinard@iro.umontreal.ca Wed Apr 10 01:31:47 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 09 Apr 2002 20:31:47 -0400 Subject: [Python-Dev] Re: Regression in unicodestr.encode()? In-Reply-To: <200204092257.g39MvRi18636@pcp742651pcs.reston01.va.comcast.net> References: <15539.16595.651881.216153@yyz.zope.com> <200204092257.g39MvRi18636@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido van Rossum] > > [Barry A. Warsaw] > > > My very limited Tim-enlightened understanding is that encoding a string > > > to UTF-8 should never produce a string with NULs. > [François] > > Besides the fact that NULs encode to themselves. In fact, 0-127 encode to > > themselves, and are never produced otherwise. Also, 254 and 255 are never > > produced, I heard that some use these to escape out within an UTF-8 string. > Hm, but isn't there a way to encode a NUL that doesn't produce a NUL? > In some variant? Not in UTF-8, that I know. The 128 characters of ASCII are just invariant, in and out, by design. Please forgive me if I'm merely repeating things that you know already, but it goes this way, looking at an UTF-8 string: 1) any byte with the eight bit cleared represents itself (so for NUL), 2) any byte with the eight bit set is part of a multi-byte sequence, 2a) any byte with both the eight bit and the seventh bit set is the start of a multi-byte sequence, 2b) any byte with the eight bit set and the seventh bit clear is the continuation of a multi-byte sequence. Point 2) indirectly says that a multi-byte sequence never holds any NUL. There is also a rule about the shortest coding. It is invalid UTF-8 to use more bytes than required, and a given UCS character has a unique UTF-8 representation. Moreover, decoders should raise an exception on non-minimal UTF-8 codings, and I do not know how Python behaves with this. The Gambit author once told me he found a way to implement the test very efficiently. One could use multi-byte sequences, that is, a sequence having no NULs, that would fool a lazy UTF-8 decoder into producing a NUL. But for this, one has to break the shortest coding rule, and start from invalid UTF-8. -- François Pinard http://www.iro.umontreal.ca/~pinard From tim.one@comcast.net Wed Apr 10 01:35:55 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 09 Apr 2002 20:35:55 -0400 Subject: [Python-Dev] Unittests In-Reply-To: <200204092326.g39NQAa18909@pcp742651pcs.reston01.va.comcast.net> Message-ID: > Tim didn't ask me before he responded; I might have suggested that > deprecating whrandom was a bit premature given your use case. Too late, Guido: the docs officially deprecated whrandom for the 2.1 release. The referenced thread took place long after that. Part of the thread that wasn't referenced pointed out that random.py supplies a whseed() method for strict bit-compatibility with whrandom.py's seed method. That seed method sucked, though, and is inappropriate for default use. The default Random.seed() is much better (for reasons spelled out in the docs). > I can certainly see the use of providing a given random algorithm for > those who want stability, while at the same time letting the random > module migrate to a stronger algorithm for the benefit for humanity. > > But then again, Tim might have a good reason why that's a bad idea. I > don't know. It's a good idea, but what was a bad idea was continuing to call the bad seed method plain "seed()". I toyed with adding a "version number" argument to seed, and that may still be a good idea; the random.py getstate() does *save* a version number, so that it's possible to change default methods entirely someday yet unpickle old random instances (2.1 or later) and get back the same generator you thought you had pickled. I ran out of time before I ran out of ideas, though . From guido@python.org Wed Apr 10 01:43:08 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 20:43:08 -0400 Subject: [Python-Dev] Developer resources (was Re: Stability and change) In-Reply-To: Your message of "Tue, 09 Apr 2002 20:20:11 EDT." <20020410002010.GA25648@panix.com> References: <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15537.55112.191870.654238@anthem.wooz.org> <15538.1515.580133.785503@12-248-41-177.client.attbi.com> <15538.2623.323978.125527@slothrop.zope.com> <20020408223655.GD562@panix.com> <15538.32008.560965.639327@grendel.zope.com> <006f01c1dfe3$8ce6c9f0$8001a8c0@COMPUTER> <15539.6606.780921.399696@grendel.zope.com> <200204091721.g39HLWm22627@odiug.zope.com> <20020410002010.GA25648@panix.com> Message-ID: <200204100043.g3A0h8P19154@pcp742651pcs.reston01.va.comcast.net> > > The problem with starship is that there are no criteria to admit new > > users. > > There wasn't any difficulty for me in getting a Starship account a bit > more than a year ago. See? That proves my theory. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Wed Apr 10 01:40:27 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 9 Apr 2002 20:40:27 -0400 Subject: [Python-Dev] NEWS consolidation In-Reply-To: <200204092329.g39NT8o18942@pcp742651pcs.reston01.va.comcast.net> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15538.194.350128.498326@12-248-41-177.client.attbi.com> <200204082310.g38NAUD14877@mercur.uphs.upenn.edu> <200204090019.g390J3C15946@pcp742651pcs.reston01.va.comcast.net> <200204091420.g39EKPF16144@mercur.uphs.upenn.edu> <20020409144717.GA7946@crystal.mems-exchange.org> <20020409230554.GC10503@panix.com> <200204092329.g39NT8o18942@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15539.35323.583803.887661@grendel.zope.com> Guido van Rossum writes: > I think that maybe the NEWS file should be turned into a database > from which different reports can be generated. This sounds good to me! > The database could be disguised as a text file in some simple format; > given the use case XML isn't high on my list of suggestions. :-) XML is never high on your list of suggestions. On the other hand, it's exactly appropriate for this task. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From tim.one@comcast.net Wed Apr 10 01:45:02 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 09 Apr 2002 20:45:02 -0400 Subject: [Python-Dev] Re: Regression in unicodestr.encode()? In-Reply-To: <200204092257.g39MvRi18636@pcp742651pcs.reston01.va.comcast.net> Message-ID: > Hm, but isn't there a way to encode a NUL that doesn't produce a NUL? > In some variant? UTF-8 has a "no \u0000 in, no NUL out" property by design (it's what makes UTF-8 uniquely well-suited to processing by crufty old 8-bit C string library routines, and that was a goal of the encoding scheme). If people are really wondering whether Barry has discovered an actual bug, don't: take his example and decode it back to Unicode. You won't get what you started with in current CVS (or at least Barry didn't when I watched him do it). That's an easier proof than indirectly wondering about UTF-8 properties. From guido@python.org Wed Apr 10 01:50:23 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 20:50:23 -0400 Subject: [Python-Dev] Re: Regression in unicodestr.encode()? In-Reply-To: Your message of "09 Apr 2002 20:31:47 EDT." References: <15539.16595.651881.216153@yyz.zope.com> <200204092257.g39MvRi18636@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204100050.g3A0oN319241@pcp742651pcs.reston01.va.comcast.net> > [Guido van Rossum] > > Hm, but isn't there a way to encode a NUL that doesn't produce a NUL? > > In some variant? [François] > There is also a rule about the shortest coding. It is invalid UTF-8 > to use more bytes than required, and a given UCS character has a > unique UTF-8 representation. Moreover, decoders should raise an > exception on non-minimal UTF-8 codings, and I do not know how Python > behaves with this. The Gambit author once told me he found a way to > implement the test very efficiently. > > One could use multi-byte sequences, that is, a sequence having no NULs, > that would fool a lazy UTF-8 decoder into producing a NUL. But for this, > one has to break the shortest coding rule, and start from invalid UTF-8. I knew all that, but I thought I'd read about a hack to encode NUL using c0 80, specifically to get around the limitation on encoded strings containing a NUL. But I can't find the reference so I'll shut up. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Apr 10 01:53:02 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 20:53:02 -0400 Subject: [Python-Dev] Unittests In-Reply-To: Your message of "Tue, 09 Apr 2002 20:35:55 EDT." References: Message-ID: <200204100053.g3A0r2x19293@pcp742651pcs.reston01.va.comcast.net> > > I can certainly see the use of providing a given random algorithm for > > those who want stability, while at the same time letting the random > > module migrate to a stronger algorithm for the benefit for humanity. > > > > But then again, Tim might have a good reason why that's a bad idea. I > > don't know. > > It's a good idea, but what was a bad idea was continuing to call the > bad seed method plain "seed()". I toyed with adding a "version > number" argument to seed, and that may still be a good idea; the > random.py getstate() does *save* a version number, so that it's > possible to change default methods entirely someday yet unpickle old > random instances (2.1 or later) and get back the same generator you > thought you had pickled. I ran out of time before I ran out of > ideas, though . I'll leave it to the OP to continue this thread. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Apr 10 01:53:50 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 20:53:50 -0400 Subject: [Python-Dev] NEWS consolidation In-Reply-To: Your message of "Tue, 09 Apr 2002 20:40:27 EDT." <15539.35323.583803.887661@grendel.zope.com> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15538.194.350128.498326@12-248-41-177.client.attbi.com> <200204082310.g38NAUD14877@mercur.uphs.upenn.edu> <200204090019.g390J3C15946@pcp742651pcs.reston01.va.comcast.net> <200204091420.g39EKPF16144@mercur.uphs.upenn.edu> <20020409144717.GA7946@crystal.mems-exchange.org> <20020409230554.GC10503@panix.com> <200204092329.g39NT8o18942@pcp742651pcs.reston01.va.comcast.net> <15539.35323.583803.887661@grendel.zope.com> Message-ID: <200204100053.g3A0rtk19316@pcp742651pcs.reston01.va.comcast.net> > XML is never high on your list of suggestions. On the other hand, > it's exactly appropriate for this task. Not if you ever want me to supply a news item again. XML is unreadable and untypable. --Guido van Rossum (home page: http://www.python.org/~guido/) From paul@prescod.net Tue Apr 9 10:02:39 2002 From: paul@prescod.net (Paul Prescod) Date: Tue, 09 Apr 2002 02:02:39 -0700 Subject: [Python-Dev] Re: Stability and change References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> <15537.43727.369744.356521@grendel.zope.com> Message-ID: <3CB2AE2F.2DAF6373@prescod.net> "Fred L. Drake, Jr." wrote: > >... > > PyXML is specifically maintaining compatibility, which means I have to > work around the lack of weakrefs when removing reference cycles. The > differences between the Python standard library and PyXML, for modules > present in both, is mostly trivial still, but the differences are > growing. ;-( > > (It's enough to make me think that Andrew may have been right about > not integrating the XML support the way we did.) I don't see the relationship between the first paragraph and the last. PyXML maintains compatibility because its maintainers care about old versions of Python. That causes your headache, whether XML support is in Python itself or not. The cure for the pain is the same as for any other third-party module: tie newer releases to newer Python versions. That said, I think that the relationship between PyXML and Python is weird too. As I said at the last conference I'd clean it up by making PyXML a package unrelated to Python XML, but I was roundly voted down. Paul Prescod From guido@python.org Wed Apr 10 02:09:54 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 21:09:54 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Tue, 09 Apr 2002 02:02:39 PDT." <3CB2AE2F.2DAF6373@prescod.net> References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> <15537.43727.369744.356521@grendel.zope.com> <3CB2AE2F.2DAF6373@prescod.net> Message-ID: <200204100109.g3A19s119435@pcp742651pcs.reston01.va.comcast.net> > The cure for the pain is the same as for any other third-party > module: tie newer releases to newer Python versions. But that's exactly what's unacceptable in theis case: newer versions of PyXML provide new PyXML features, and Python users who wish to use the new PyXML features should *not* have to upgrade to the latest and greatest Python. That would cause the kind of dependency nightmares we just heard about with CPAN (and which are also common with RPMs in my experience -- I recently had to upgrade a printer driver and it was a true nightmare). --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Wed Apr 10 02:13:37 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 09 Apr 2002 21:13:37 -0400 Subject: [Python-Dev] Re: Regression in unicodestr.encode()? In-Reply-To: <200204100050.g3A0oN319241@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > I knew all that, but I thought I'd read about a hack to encode NUL > using c0 80, specifically to get around the limitation on encoded > strings containing a NUL. Ah, that violates the "shortest encoding" rule, so is invalid UTF-8. I'm sure people have done it, though, and that many UTF-8 encoders accept it. Python's doesn't: >>> unicode('\xc0\x80', 'utf-8') Traceback (most recent call last): File "", line 1, in ? UnicodeError: UTF-8 decoding error: illegal encoding >>> Believe it or not, accepting non-shortest encodings is considered to be "a security hole"(!). That's a sad story of its own ... From Anthony Baxter Wed Apr 10 02:47:15 2002 From: Anthony Baxter (Anthony Baxter) Date: Wed, 10 Apr 2002 11:47:15 +1000 Subject: [Python-Dev] NEWS consolidation In-Reply-To: Message from Guido van Rossum of "Tue, 09 Apr 2002 20:53:50 -0400." <200204100053.g3A0rtk19316@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204100147.g3A1lF018490@mbuna.arbhome.com.au> >>> Guido van Rossum wrote > > XML is never high on your list of suggestions. On the other hand, > > it's exactly appropriate for this task. > > Not if you ever want me to supply a news item again. XML is > unreadable and untypable. what about using LaTeX? :) From barry@zope.com Wed Apr 10 02:58:05 2002 From: barry@zope.com (Barry A. Warsaw) Date: Tue, 9 Apr 2002 21:58:05 -0400 Subject: [Python-Dev] Regression in unicodestr.encode()? References: <15539.16595.651881.216153@yyz.zope.com> Message-ID: <15539.39981.221392.757470@anthem.wooz.org> I've submitted a SF bug on this and assigned it to MAL. http://sourceforge.net/tracker/index.php?func=detail&aid=541828&group_id=5470&atid=105470 -Barry From andy@reportlab.com Wed Apr 10 03:03:25 2002 From: andy@reportlab.com (Andy Robinson) Date: Wed, 10 Apr 2002 03:03:25 +0100 Subject: [Python-Dev] Stability and Change Message-ID: I missed this thread due to a heavy schedule and hope people won't mind one more comment. Let's be really blunt: Frequent releases cost me money. I am sure I am not alone. Like many small firms we have to trade off uses of our time 3 ways between: (a) billing customers for revenue to live off, (b) constantly re-testing, building binaries and installers etc etc. just to keep it all working and everyone supported (c) doing what we really want to - working on product features. When I started ReportLab I never realized (b) could be so much work and how little time would be left for (c) as a result. I desperately want to see a "recommended stable version" declared with intervals of 18 months or greater. A very important point for me is how often the C API changes - that's when you have to build new binaries for everything, retest all existing apps, and then spend a lot of time which is hard to charge walking your customers through a big upgrade cycle. I could write an essay about how much crap can be involved in this. Do you know how much advance planning it takes to get a large customer to upgrade a package on their development, user test and live boxes, when they have no perceived business need for it? Ironically it seems to be about six months :-) 24 months would be even better. In the meantime there is lots of time to try out new language features, even get them wrong and retract them, and I'll be playing with them in the evenings too - just not on customer sites. Meanwhile, those of us who regularly build shared libraries and installers for various platforms would just stick on the "recommended stable version" and not feel pressurised into doing binary installers of the ones in between. By definition this version will have to have existed for a little while before it is so designated, and there should be some commitment to fixing bugs in it. For me this is not about code stability, which is generally excellent. It's about giving the commercial world a schedule they can make plans around. I believe this designation will save time for everyone, including Pythonlabs. Best Regards, Andy Robinson ReportLab Inc. From jeremy@zope.com Wed Apr 10 02:59:57 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Tue, 9 Apr 2002 21:59:57 -0400 Subject: [Python-Dev] NEWS consolidation In-Reply-To: <200204100053.g3A0rtk19316@pcp742651pcs.reston01.va.comcast.net> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15538.194.350128.498326@12-248-41-177.client.attbi.com> <200204082310.g38NAUD14877@mercur.uphs.upenn.edu> <200204090019.g390J3C15946@pcp742651pcs.reston01.va.comcast.net> <200204091420.g39EKPF16144@mercur.uphs.upenn.edu> <20020409144717.GA7946@crystal.mems-exchange.org> <20020409230554.GC10503@panix.com> <200204092329.g39NT8o18942@pcp742651pcs.reston01.va.comcast.net> <15539.35323.583803.887661@grendel.zope.com> <200204100053.g3A0rtk19316@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15539.40093.108510.543588@slothrop.zope.com> >>>>> "GvR" == Guido van Rossum writes: >> XML is never high on your list of suggestions. On the other >> hand, it's exactly appropriate for this task. GvR> Not if you ever want me to supply a news item again. XML is GvR> unreadable and untypable. So what about reST? From neal@metaslash.com Wed Apr 10 03:03:58 2002 From: neal@metaslash.com (Neal Norwitz) Date: Tue, 09 Apr 2002 22:03:58 -0400 Subject: random vs whrandom (was RE: [Python-Dev] Unittests) References: Message-ID: <3CB39D8E.C5E837CE@metaslash.com> Tim Peters wrote: > > >>>> whrandom.py > > >>> This is a deprecated module, no? > > As its docs say, "Deprecated since release 2.1. Use random instead." > Somebody toss in DeprecationWarnings or it will never go away. I added a deprecation warning, but Fred beat me to it. I added warnings to statcache & xmllib which are deprecated in the docs. Any others need to be done? Should statcache be removed from test_sundry? Should any modules be moved into lib-old? posixfile? Neal From pinard@iro.umontreal.ca Wed Apr 10 03:05:48 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 09 Apr 2002 22:05:48 -0400 Subject: [Python-Dev] Re: NEWS consolidation In-Reply-To: <200204100053.g3A0rtk19316@pcp742651pcs.reston01.va.comcast.net> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15538.194.350128.498326@12-248-41-177.client.attbi.com> <200204082310.g38NAUD14877@mercur.uphs.upenn.edu> <200204090019.g390J3C15946@pcp742651pcs.reston01.va.comcast.net> <200204091420.g39EKPF16144@mercur.uphs.upenn.edu> <20020409144717.GA7946@crystal.mems-exchange.org> <20020409230554.GC10503@panix.com> <200204092329.g39NT8o18942@pcp742651pcs.reston01.va.comcast.net> <15539.35323.583803.887661@grendel.zope.com> <200204100053.g3A0rtk19316@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido van Rossum] > XML is unreadable and untypable. Well, Lennart Staflin's superb PSGML package (for Emacs) makes SGML rather easy to type, and I guess it would make XML tractable as well. Without Emacs and PSGML, I would guess that typing XML is not easy. I would surely make myself something if I had to. Much granted that XML is rebuking for the human reader, the information drawns under the noise. When "they" decided to design XML out of SGML, they compromised a bit too far towards machines, and away from humans. Nevertheless, and surprisingly to me, XML has been extremely successful. My reluctance more or less vanished (but not my opinion!) after I used Glade, together with James Henstridge's libglade, of course! :-) -- François Pinard http://www.iro.umontreal.ca/~pinard From guido@python.org Wed Apr 10 03:21:46 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 22:21:46 -0400 Subject: [Python-Dev] NEWS consolidation In-Reply-To: Your message of "Tue, 09 Apr 2002 21:59:57 EDT." <15539.40093.108510.543588@slothrop.zope.com> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15538.194.350128.498326@12-248-41-177.client.attbi.com> <200204082310.g38NAUD14877@mercur.uphs.upenn.edu> <200204090019.g390J3C15946@pcp742651pcs.reston01.va.comcast.net> <200204091420.g39EKPF16144@mercur.uphs.upenn.edu> <20020409144717.GA7946@crystal.mems-exchange.org> <20020409230554.GC10503@panix.com> <200204092329.g39NT8o18942@pcp742651pcs.reston01.va.comcast.net> <15539.35323.583803.887661@grendel.zope.com> <200204100053.g3A0rtk19316@pcp742651pcs.reston01.va.comcast.net> <15539.40093.108510.543588@slothrop.zope.com> Message-ID: <200204100221.g3A2Llx19626@pcp742651pcs.reston01.va.comcast.net> > >> XML is never high on your list of suggestions. On the other > >> hand, it's exactly appropriate for this task. > > GvR> Not if you ever want me to supply a news item again. XML is > GvR> unreadable and untypable. > > So what about reST? Probably overkill in this case, although it would allow nicer HTML to be generated e.g. for the release notes on the web. I was thinking of using the entry format currently used (a paragraph with a leading hyphen) and a little bit of meta-data, e.g.: - release number - category (e.g. Core and builtins, Extension modules, Library, etc.). If you want to get fancy, add: - timestamp (could be added automatically when you add the entry) - optional list of SF bug/patch numbers. If you really want to get fancy, some kind of linkage with CVS checkin messages might be cute, but usually there's no 1-1 correspondence between checkins and NEWS items. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Apr 10 03:23:57 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 22:23:57 -0400 Subject: random vs whrandom (was RE: [Python-Dev] Unittests) In-Reply-To: Your message of "Tue, 09 Apr 2002 22:03:58 EDT." <3CB39D8E.C5E837CE@metaslash.com> References: <3CB39D8E.C5E837CE@metaslash.com> Message-ID: <200204100223.g3A2Nvp19644@pcp742651pcs.reston01.va.comcast.net> > I added warnings to statcache & xmllib which are deprecated in the docs. > Any others need to be done? Should statcache be removed from test_sundry? > Should any modules be moved into lib-old? posixfile? A good exercise would be to do a 1-minute review of every module in the library and decide for yourself if you'd deprecate it if you were BDFL. I'll gladly review the list. --Guido van Rossum (home page: http://www.python.org/~guido/) From greg@cosc.canterbury.ac.nz Wed Apr 10 03:29:02 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Wed, 10 Apr 2002 14:29:02 +1200 (NZST) Subject: [Python-Dev] Re: ANN: Pyrex - a language for writing Python extension modules In-Reply-To: <15532.51903.323557.147303@12-248-41-177.client.attbi.com> Message-ID: <200204100229.OAA00739@s454.cosc.canterbury.ac.nz> Skip Montanaro : > Interesting, but why not simply attach the type information to function > attributes so the Pyrex code really is just Python? I imagine that would look extremely ugly. Also, functions aren't the only places that type information, and other kinds of information, is needed in Pyrex. Have you read the Language Overview? There's much, much more to Pyrex than just annotating things with types. > You can defer Pyrex compilation until you realize you need it. Naively > compiling all your code to C makes for a lot of (object) code bloat and > doesn't necessarily speed anything up (if, for example, the code is I/O > bound or not executed frequently enough). This was one of the problems > with Python2C. You're not meant to compile all your Python code with Pyrex. You only put Python code in a Pyrex module if it's needed to interface with non-Python code. Pyrex isn't designed to speed up Python code, but to bridge the gap between Python and non-Python. Speed, if any, comes from the non-Python code (Pyrex code operating on C data, or external C code). Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From jepler@unpythonic.dhs.org Wed Apr 10 03:46:11 2002 From: jepler@unpythonic.dhs.org (jepler@unpythonic.dhs.org) Date: Tue, 9 Apr 2002 21:46:11 -0500 Subject: [Python-Dev] Re: Regression in unicodestr.encode()? In-Reply-To: <200204100050.g3A0oN319241@pcp742651pcs.reston01.va.comcast.net> References: <15539.16595.651881.216153@yyz.zope.com> <200204092257.g39MvRi18636@pcp742651pcs.reston01.va.comcast.net> <200204100050.g3A0oN319241@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020409214611.B1356@unpythonic.dhs.org> On Tue, Apr 09, 2002 at 08:50:23PM -0400, Guido van Rossum wrote: > I knew all that, but I thought I'd read about a hack to encode NUL > using c0 80, specifically to get around the limitation on encoded > strings containing a NUL. But I can't find the reference so I'll shut > up. Tcl does, even including a CVS checkout from a few weeks ago. It's done deliberately, as though some internal APIs didn't handle NUL-containing strings correctly. I am certain that I saw a paper about precisely this detail of tcl, but apparently it's been taken down in shame. I did find: TCL does its best to accept anything, but produce only shortest-form output. The one special case is embedded nulls (0x0000), where Tcl produces 0xC0 0x80 in order to avoid possible null-termination problems with non-UTF aware code. It probably wouldn't break anything to to disallow non-shortest form UTF-8 for all but this one case. If you eliminate the 0xc080 case, you'll have to check to make sure *everything* is length encoded. -- http://mail.nl.linux.org/linux-utf8/2001-03/msg00029.html About Java: The interfaces java.io.DataInput and java.io.DataOutput have methods called `readUTF' and `writeUTF' respectively. But note that they don't use UTF-8; they use a modified UTF-8 encoding: the NUL character is encoded as the two-byte sequence 0xC0 0x80 instead of 0x00, and a 0x00 byte is added at the end. Encoded this way, strings can contain NUL characters and nevertheless need not be prefixed with a length field - the C functions like strlen() and strcpy() can be used to manipulate them. -- http://www.tldp.org/HOWTO/Unicode-HOWTO-6.html Why Python refuses to do it this way: for security reasons, the UTF-8 codec gives you an "illegal encoding" error in this case. -- http://aspn.activestate.com/ASPN/Mail/Message/i18n-sig/581440 (our very own Mr. Fredrik Lundh, also quoting the Gospel of RFC, chapter 2279) Ah, and here's the article I originally found the c0 80 idea presented as a way to make existing programs handle embedded NULs: Now going the other way. In orthodox UTF-8, a NUL byte(\x00) is represented by a NUL byte. Plain enough. But in Tcl we sometimes want NUL bytes inside "binary" strings (e.g. image data), without them terminating it as a real NUL byte does. To represent a NUL byte without any physical NUL bytes, we treat it like a character above ASCII, which must be a minimum two bytes long: (110)00000 (10)000000 => C0 80 Whoops. Took us a while, but now we can read UTF-8, bit by bit. -- http://mini.net/tcl/1211.html I'm terribly glad that Python has gotten this detail right. Jeff From guido@python.org Wed Apr 10 04:06:12 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 23:06:12 -0400 Subject: [Python-Dev] Stability and Change In-Reply-To: Your message of "Wed, 10 Apr 2002 03:03:25 BST." References: Message-ID: <200204100306.g3A36CE20294@pcp742651pcs.reston01.va.comcast.net> > I missed this thread due to a heavy schedule > and hope people won't mind one more comment. > Let's be really blunt: > > Frequent releases cost me money. [...] > I desperately want to see a "recommended stable version" declared > with intervals of 18 months or greater. [...] > 24 months would be even better. Thanks for a message from the trenches. (Alex can be really convincing, but I'm never quite sure if he's *really* been in trenches or if he's just a damn good storyteller. With Andy, I *know* he talks no bullshit.) I've heard "18 months or more" from several (presumably) independent sources now. I'd like to give you that, but for me, as a developer, it feels like an eternity, and I'm more comfortable with 6-8 months for major releases (of the 2.2 kind). How about this as a compromise: each major release (which for Python is 2.0, 2.1, 2.2, 2.3 etc., never mind that only the "minor" digit varies -- that's how we number 'em) has a shelf-life of at least three subsequent major releases or 24 months, whichever is shorter, but never less than 18 months. Shelf-life means that we follow up with bugfix releases (like 2.1.1, 2.1.2, 2.1.3) to maintain and improve stability, if necessary back-porting selected minor features, but with the absolute guarantee that (a) the binary API doesn't change, so C extensions are completely portable between bugfix releases, and (b) no old code break unless it depends on bugs being present or names being absent from any given namespace. (The latter qualification means that a bugfix release may add methods to classes, functions to modules, new modules or packages, and even built-in names; but it cannot take away, and it cannot change function signatures except by adding optional arguments. I believe this leeway is necessary to be able to add minor features.) This means that once you pick a release, you can rest in peace for at least 18-24 months, without feeling left behind. You may have to teach your customers *not* to upgrade to the latest and greatest major release (but that doesn't seem to be a problem :-), but they should be comfortable picking up a bugfix release any time they feel like it without worrying about breaking something. More likely, they would only upgrade when you recommend that they do so (e.g. because you know it addresses a specific problem they've got), but these upgrades should be very smooth and free of surprises. (We've got some experience with that now with Zope users and various versions on the 2.1.x axis; it seems to work well.) It may mean that you have to reel in your own developers a bit, since they have to develop for whatever Python release you've picked, so they may not be able to use the generators / bools / whatever they'd like to use. But that seems a minor inconvenience. I understand that a feature in the latest major release is harder to do without than a feature in CVS, but again, Zope current practice suggests that it's no big deal. Comments? --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@alum.mit.edu Wed Apr 10 04:04:18 2002 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Tue, 09 Apr 2002 23:04:18 -0400 (EDT) Subject: [Python-Dev] test_mpz skipped? Message-ID: <0GUC007KO0J5VT@mtaout02.icomcast.net> I just got the following test output on Linux. > test test_mpz skipped -- No module named mpz > 1 test skipped: > test_mpz > 1 skip unexpected on linux2: > test_mpz Is this just a question of adding test_mpz to the list of expected skips on Linux? Or is there some problem with my build? Jeremy From neal@metaslash.com Wed Apr 10 04:10:22 2002 From: neal@metaslash.com (Neal Norwitz) Date: Tue, 09 Apr 2002 23:10:22 -0400 Subject: [Python-Dev] Re: test_mpz skipped? References: <0GUC007KO0J5VT@mtaout02.icomcast.net> Message-ID: <3CB3AD1E.D748A8B8@metaslash.com> Jeremy Hylton wrote: > > I just got the following test output on Linux. > > > test test_mpz skipped -- No module named mpz > > 1 test skipped: > > test_mpz > > 1 skip unexpected on linux2: > > test_mpz > > Is this just a question of adding test_mpz to the list of expected > skips on Linux? Or is there some problem with my build? It works on my box (redhat 7.2). There is an Modules/mpzmodule.c which uses the GNU MP (multi-precision) library. libmp.a would need to be on your box. It's in gmp-devel-3.1.1-4 on mine. It's possible other distro's don't include MP. I'm fine with adding it to the skips list. Neal From guido@python.org Wed Apr 10 04:15:13 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Apr 2002 23:15:13 -0400 Subject: [Python-Dev] test_mpz skipped? In-Reply-To: Your message of "Tue, 09 Apr 2002 23:04:18 EDT." <0GUC007KO0J5VT@mtaout02.icomcast.net> References: <0GUC007KO0J5VT@mtaout02.icomcast.net> Message-ID: <200204100315.g3A3FD520380@pcp742651pcs.reston01.va.comcast.net> > I just got the following test output on Linux. > > > test test_mpz skipped -- No module named mpz > > 1 test skipped: > > test_mpz > > 1 skip unexpected on linux2: > > test_mpz > > Is this just a question of adding test_mpz to the list of expected > skips on Linux? Yes. It depends on a 3rd party library that's not commonly present. > Or is there some problem with my build? Who knows. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Wed Apr 10 04:18:57 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 09 Apr 2002 23:18:57 -0400 Subject: [Python-Dev] "Unstable" is an ambiguous word... In-Reply-To: <200204091455.g39Etk608602@mbuna.arbhome.com.au> Message-ID: [Michael Hudson] > What about recent pymalloc changes? They should yield some > performance enhancements, will make no difference to Python code, but > probably require cooperation from extension authors... [Anthony Baxter] > Aren't they binary-incompatible? In which case, thanks but no thanks, > for me binary compatibility is one of the fundamental parts of a minor > release. No, we're enduring several rings of hell to ensure not only that the new pymalloc is binary compatible, but also that it continues to "support" any number of insane usages found in real life code, including insane usage officially deprecated in old docs. This is increasing the time needed to get pymalloc "finished" by at least a factor of 100000 . From paul@prescod.net Wed Apr 10 04:32:10 2002 From: paul@prescod.net (Paul Prescod) Date: Tue, 09 Apr 2002 20:32:10 -0700 Subject: [Python-Dev] Re: Stability and change References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> <15537.43727.369744.356521@grendel.zope.com> <3CB2AE2F.2DAF6373@prescod.net> <200204100109.g3A19s119435@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CB3B23A.2277F6CE@prescod.net> Guido van Rossum wrote: > > > The cure for the pain is the same as for any other third-party > > module: tie newer releases to newer Python versions. > > But that's exactly what's unacceptable in theis case: newer versions > of PyXML provide new PyXML features, and Python users who wish to use > the new PyXML features should *not* have to upgrade to the latest and > greatest Python. I still don't see how this is any different than any other Python module. It's probably painful keeping NumPy compatible with old versions of Python also. How is PyXML unique in this? Paul Prescod From DavidA@ActiveState.com Wed Apr 10 04:36:11 2002 From: DavidA@ActiveState.com (David Ascher) Date: Tue, 09 Apr 2002 20:36:11 -0700 Subject: [Python-Dev] NEWS consolidation References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15538.194.350128.498326@12-248-41-177.client.attbi.com> <200204082310.g38NAUD14877@mercur.uphs.upenn.edu> <200204090019.g390J3C15946@pcp742651pcs.reston01.va.comcast.net> <200204091420.g39EKPF16144@mercur.uphs.upenn.edu> <20020409144717.GA7946@crystal.mems-exchange.org> <20020409230554.GC10503@panix.com> <200204092329.g39NT8o18942@pcp742651pcs.reston01.va.comcast.net> <15539.35323.583803.887661@grendel.zope.com> <200204100053.g3A0rtk19316@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CB3B32B.677A3C1D@ActiveState.com> Guido van Rossum wrote: > > > XML is never high on your list of suggestions. On the other hand, > > it's exactly appropriate for this task. > > Not if you ever want me to supply a news item again. XML is > unreadable and untypable. Think of XML as a database format. The input could be in a formalized version of the current NEWS format, and the output could be whatever people wanted to write. It's not a bad database format for textual data. Better than pickles, IMO. --da From skip@pobox.com Wed Apr 10 05:15:12 2002 From: skip@pobox.com (Skip Montanaro) Date: Tue, 9 Apr 2002 23:15:12 -0500 Subject: [Python-Dev] Stability and Change In-Reply-To: <200204100306.g3A36CE20294@pcp742651pcs.reston01.va.comcast.net> References: <200204100306.g3A36CE20294@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15539.48208.510028.637885@12-248-41-177.client.attbi.com> >> Frequent releases cost me money. ... >> I desperately want to see a "recommended stable version" declared >> with intervals of 18 months or greater. ... >> 24 months would be even better. Guido> I've heard "18 months or more" from several (presumably) Guido> independent sources now. I'd like to give you that, but for me, Guido> as a developer, it feels like an eternity, and I'm more Guido> comfortable with 6-8 months for major releases (of the 2.2 kind). Guido> How about this as a compromise: each major release (which for Guido> Python is 2.0, 2.1, 2.2, 2.3 etc., never mind that only the Guido> "minor" digit varies -- that's how we number 'em) has a Guido> shelf-life of at least three subsequent major releases or 24 Guido> months, whichever is shorter, but never less than 18 months. Guido> Shelf-life means that we follow up with bugfix releases (like Guido> 2.1.1, 2.1.2, 2.1.3) to maintain and improve stability, if Guido> necessary back-porting selected minor features, but with the Guido> absolute guarantee that (a) the binary API doesn't change, so C Guido> extensions are completely portable between bugfix releases, and Guido> (b) no old code break unless it depends on bugs being present or Guido> names being absent from any given namespace. ... Guido> Comments? Guido, >From where I sit this seems untenable, simply because what you're proposing is going to require having three or four release managers at the same time, each managing bugfixes and (what was the term?) "minor features" for a previous release. Ain't no way I'm going to backport a change to three previous releases. I can barely manage getting the occasional patch or bugfix mostly correct on the trunk, and I suspect that other peoples' accuracy will diminish as they have to backport patches (possibly with subtle code rearrangement) to multiple previous releases. By proposing this, all you're doing is making more work for the people on the SF developers' list. While I'm sure this probably won't sit well with Andy and other commercial folks listening, I'm going to suggest that it's wrong for ReportLabs to expect the Python community as a whole to support their desire to run old versions of Python. I don't get a penny from the work I put into the Python source tree. Meager though my effort may be, it is still contributed time. Andy at least has the prospect of billing a client to cover (some of) his time. While that may cut his profit margins, I don't think it's unreasonable to expect ReportLabs and other companies that want to continue to run old releases to pay for that privilege one way or another. They can step up to the bar a couple different ways, either taking on some of the labor themselves or contracting it out (possibly by supporting someone in the community who would function as a release manager for Python version(s) of interest). where-can-i-send-the-invoice?-ly, y'rs, Skip From andymac@bullseye.apana.org.au Tue Apr 9 23:17:45 2002 From: andymac@bullseye.apana.org.au (Andrew MacIntyre) Date: Wed, 10 Apr 2002 09:17:45 +1100 (edt) Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15538.8494.31517.706412@12-248-41-177.client.attbi.com> Message-ID: On Mon, 8 Apr 2002, Skip Montanaro wrote: > Chris> The double-branch option is easier, and I know that when the > Chris> FreeBSD group releases 4.5-RELEASE, it's been tested and solid, > Chris> and will be supported by bug fixes for some period of time. They > Chris> have only just recently stopped issue security patches for the > Chris> 2.x tree. > > Is *-RELEASE a third branch, or is it what falls out the end of the *-STABLE > pipe? RELEASE is a release tag, STABLE is a branch name. -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au | Snail: PO Box 370 andymac@pcug.org.au | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From andymac@bullseye.apana.org.au Tue Apr 9 23:12:03 2002 From: andymac@bullseye.apana.org.au (Andrew MacIntyre) Date: Wed, 10 Apr 2002 09:12:03 +1100 (edt) Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> Message-ID: On Mon, 8 Apr 2002, Skip Montanaro wrote: > Windows installers should not be created for > development releases and Linux distributions will have no business > installing a Python development release in /usr. Not creating some installable Windows package for development releases cuts of most of your potential reviewers on that platform. -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au | Snail: PO Box 370 andymac@pcug.org.au | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From jafo@tummy.com Wed Apr 10 05:17:18 2002 From: jafo@tummy.com (Sean Reifschneider) Date: Tue, 9 Apr 2002 22:17:18 -0600 Subject: [Python-Dev] 2.2.1c2 RPMs In-Reply-To: ; from mwh@python.net on Tue, Apr 09, 2002 at 10:41:31AM +0100 References: <20020408235227.U16962@tummy.com> Message-ID: <20020409221718.X11182@tummy.com> On Tue, Apr 09, 2002 at 10:41:31AM +0100, Michael Hudson wrote: >2:00-5:00 You grab the above, build some RPMs, and leave them in > creosote.python.org:~jafo/python-2.2.1-foo.rpm. Emailing me > md5s probably gets you brownie points. I sleep. > >11:00 I move all files into place on creosote, edit web pages, write an > announcement, etc. I go nowhere near sourceforge (hurrah!). I've made a bunch of changes to the 2.2.1 pages relating largely to the RPMS. I also fixed a couple of typos, and changed the string the link in the 2.2 pages to point to the 2.2.1. I tweeked the structure of the rpms FTP directory, the new structure is under creosote:~jafo/rpm-ftp -- it includes directories for the different platforms. I'm happy with the current look of the RPMs page. Oh, I also added the link in the nav-bar-thing to include RPMs (it was commented out, I don't know if this was intentional for the final release, but I think it should be there). So, you should just be able to move the rpms-ftp directory over to ftp/python/2.2.1/rpms and it should be golden. The md5sums with the new structure are: 77de2a345594876ea7b138780200b507 ./KRUD-GPG-KEY fc1259287ed5ee4cec8b1ba01a74eb62 ./mdk8.2/python2-2.2.1-1.i586.rpm c791d31363cc97e7bcd59bd543deccfa ./mdk8.2/python2-devel-2.2.1-1.i586.rpm f58f622eec85407f94bc49a766328217 ./mdk8.2/python2-docs-2.2.1-1.i586.rpm 5bdc73c043a666f02c27e13c42f9b13b ./mdk8.2/python2-tkinter-2.2.1-1.i586.rpm 8967b7f4f20e7192e76ec85e7b723216 ./mdk8.2/python2-tools-2.2.1-1.i586.rpm b0f333d0676580d872454d5b56d06db6 ./rh6.2/python2-2.2.1-1.i386.rpm 41b8aa98b31e8b7a7182bb0c0d67a8b5 ./rh6.2/python2-devel-2.2.1-1.i386.rpm ba5f6b36bb8df9d9ca249448a1d07b25 ./rh6.2/python2-docs-2.2.1-1.i386.rpm 067d782d8d1416ad7082ab9b6673d5d1 ./rh6.2/python2-tkinter-2.2.1-1.i386.rpm caa4e92783fbab5bb548f6c1b480950d ./rh6.2/python2-tools-2.2.1-1.i386.rpm fe04c9397155a952174150c81452327c ./rh7.2/python2-2.2.1-1.i386.rpm 4ada9feaf97521140909c843f7d97ef1 ./rh7.2/python2-devel-2.2.1-1.i386.rpm 9ae59f01ef3c7741faa4abf5feaad221 ./rh7.2/python2-docs-2.2.1-1.i386.rpm 22f11a28dfe540f3e4fa69e335629d94 ./rh7.2/python2-tkinter-2.2.1-1.i386.rpm c0909fd8880059acf17c1a92402244bc ./rh7.2/python2-tools-2.2.1-1.i386.rpm 3c507b8fa8960cff0131c88bfc5fdaed ./python2-2.2.1-1.src.rpm Sean -- Reading computer manuals without the hardware is as frustrating as reading sex manuals without the software. -- Arthur C. Clarke Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From aahz@pythoncraft.com Wed Apr 10 05:19:23 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 10 Apr 2002 00:19:23 -0400 Subject: [Python-Dev] Developer resources (was Re: Stability and change) In-Reply-To: <200204100043.g3A0h8P19154@pcp742651pcs.reston01.va.comcast.net> References: <15537.55112.191870.654238@anthem.wooz.org> <15538.1515.580133.785503@12-248-41-177.client.attbi.com> <15538.2623.323978.125527@slothrop.zope.com> <20020408223655.GD562@panix.com> <15538.32008.560965.639327@grendel.zope.com> <006f01c1dfe3$8ce6c9f0$8001a8c0@COMPUTER> <15539.6606.780921.399696@grendel.zope.com> <200204091721.g39HLWm22627@odiug.zope.com> <20020410002010.GA25648@panix.com> <200204100043.g3A0h8P19154@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020410041923.GA5501@panix.com> On Tue, Apr 09, 2002, Guido van Rossum wrote: > >>> The problem with starship is that there are no criteria to admit new >>> users. >> >> There wasn't any difficulty for me in getting a Starship account a bit >> more than a year ago. > > See? That proves my theory. :-) And how is that "a problem"? Are you worried about hordes of people descending on Starship? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From skip@pobox.com Wed Apr 10 05:25:42 2002 From: skip@pobox.com (Skip Montanaro) Date: Tue, 9 Apr 2002 23:25:42 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: <15538.8494.31517.706412@12-248-41-177.client.attbi.com> Message-ID: <15539.48838.254598.710229@12-248-41-177.client.attbi.com> >> Is *-RELEASE a third branch, or is it what falls out the end of the >> *-STABLE pipe? Andrew> RELEASE is a release tag, STABLE is a branch name. How do they relate to RELENG? Skip From skip@pobox.com Wed Apr 10 05:31:39 2002 From: skip@pobox.com (Skip Montanaro) Date: Tue, 9 Apr 2002 23:31:39 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> Message-ID: <15539.49195.297507.616138@12-248-41-177.client.attbi.com> >> Windows installers should not be created for development releases and >> Linux distributions will have no business installing a Python >> development release in /usr. Andrew> Not creating some installable Windows package for development Andrew> releases cuts of most of your potential reviewers on that Andrew> platform. Oh well. Guido already complains (rightfully so, I believe) that there is very little community input on the existing alpha/beta/rc1/rc2 process. As he put it earlier today, people who check out 2.2final think they are early adopters. If most people don't test out early releases now, Tim's been wasting his time generating all those installers. The only way a significant change in the development model is going to work is if the effort necessary to generate micro releases drops drastically. If churning out a binary installer *suitable for a development branch* can be reduced to a few minutes, fine. If not, then I suspect either someone else will have to take that task over from Tim or Windows installers won't get cut for development point releases. Skip From aahz@pythoncraft.com Wed Apr 10 05:36:29 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 10 Apr 2002 00:36:29 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15539.49195.297507.616138@12-248-41-177.client.attbi.com> References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> <15539.49195.297507.616138@12-248-41-177.client.attbi.com> Message-ID: <20020410043629.GA6483@panix.com> On Tue, Apr 09, 2002, Skip Montanaro wrote: > > The only way a significant change in the development model is going to work > is if the effort necessary to generate micro releases drops drastically. If > churning out a binary installer *suitable for a development branch* can be > reduced to a few minutes, fine. If not, then I suspect either someone else > will have to take that task over from Tim or Windows installers won't get > cut for development point releases. Seems to me that for dev releases, there's no need for a "Windows installer". A zip file that contains the necessary install tree ought to be sufficient. There will still need to be the effort of convincing people to download those releases, but I think (based on *my* mindset) that it will actually be easier because there won't be any worries about mucking with the registry. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From greg@cosc.canterbury.ac.nz Wed Apr 10 05:44:42 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Wed, 10 Apr 2002 16:44:42 +1200 (NZST) Subject: [Python-Dev] Re: About Pyrex In-Reply-To: <200204081738.g38Hcfm512498@boa.lbl.gov> Message-ID: <200204100444.QAA00833@s454.cosc.canterbury.ac.nz> > I was wondering if you know about Boost.Python: > http://www.python.org/sigs/c++-sig/ I had heard of it, but hadn't looked very closely at it. It looks interesting -- you seem to be approaching the problem from exactly the opposite direction, in a way. It's hard to get an idea of how your approach stacks up against mine, because they're so different. One thing I can say is that yours stands a better chance of coping with C++ in its full generality than mine does at the moment! > In your example you cross the boundary (via a C API call) each time > you produce a result. Isn't that in general eliminating much of the > gain of using C? No, because in that example there's a lot of computation going on between each result. The inner loop there isn't the one collecting results, it's the one doing the divisions, which runs entirely in C. I hope to teach Pyrex more about some of the builtin objects, so, e.g. you could return an array.array of numbers instead of a Python list. But even then, if Python code is to do anything with the individual numbers, they will get converted to Python objects. In general -- if Python code needs to get at the results, they have to be converted to Python objects somewhere, somehow. I don't see a way around that. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From DavidA@ActiveState.com Wed Apr 10 05:46:32 2002 From: DavidA@ActiveState.com (David Ascher) Date: Tue, 09 Apr 2002 21:46:32 -0700 Subject: [Python-Dev] Re: Stability and change References: Message-ID: <3CB3C3A8.136A5E0C@ActiveState.com> Andrew MacIntyre wrote: > > On Mon, 8 Apr 2002, Skip Montanaro wrote: > > > Windows installers should not be created for > > development releases and Linux distributions will have no business > > installing a Python development release in /usr. > > Not creating some installable Windows package for development releases > cuts of most of your potential reviewers on that platform. An idea that I've been kicking around internally is to have ActivePython 'CVS' builds made available on a periodic and automated basis, to help test what we used to call "the CVS version". More details will be forthcoming. --david ascher From haering_python@gmx.de Wed Apr 10 05:48:11 2002 From: haering_python@gmx.de (Gerhard =?iso-8859-15?Q?H=E4ring?=) Date: Wed, 10 Apr 2002 06:48:11 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> Message-ID: <20020410044810.GA4932@lilith.my-fqdn.de> * Andrew MacIntyre [2002-04-10 09:12 +1100]: > On Mon, 8 Apr 2002, Skip Montanaro wrote: > > > Windows installers should not be created for > > development releases and Linux distributions will have no business > > installing a Python development release in /usr. > > Not creating some installable Windows package for development releases > cuts of most of your potential reviewers on that platform. I doubt that there's a lot of input from Windows users for alphas/betas for major releases. The reason is that if they needed to test their software, they'd also need new builds of the third-party packages they use on Windows. But some of these are hard to build and most users don't know there are perfectly good free compilers for Windows. If it is easier for Tim/the Pythonlabs folks, why not just (for development snapshorts) put the Windows binaries in a zipfile that can be unzipped and will just work? I fail to see why an installer is absolutely necessary for Windows users. Gerhard -- This sig powered by Python! Außentemperatur in München: 1.6 °C Wind: 2.5 m/s From DavidA@ActiveState.com Wed Apr 10 05:48:21 2002 From: DavidA@ActiveState.com (David Ascher) Date: Tue, 09 Apr 2002 21:48:21 -0700 Subject: [Python-Dev] blending threads Message-ID: <3CB3C415.3BF27BA5@ActiveState.com> People who do significant grunt work on the CVS tree backporting fixes automatically get starship accounts. Presto, solve two problems with one rule. =). From skip@pobox.com Wed Apr 10 05:59:38 2002 From: skip@pobox.com (Skip Montanaro) Date: Tue, 9 Apr 2002 23:59:38 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <20020410043629.GA6483@panix.com> References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> <15539.49195.297507.616138@12-248-41-177.client.attbi.com> <20020410043629.GA6483@panix.com> Message-ID: <15539.50874.831225.914697@12-248-41-177.client.attbi.com> >> If churning out a binary installer *suitable for a development >> branch* can be reduced to a few minutes, fine. If not ... aahz> Seems to me that for dev releases, there's no need for a "Windows aahz> installer". A zip file that contains the necessary install tree aahz> ought to be sufficient. I think the contention was implicitly that many people who might be able to help test development releases couldn't because they lack the necessary tools to build from source. Skip From aahz@pythoncraft.com Wed Apr 10 05:59:56 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 10 Apr 2002 00:59:56 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15539.50874.831225.914697@12-248-41-177.client.attbi.com> References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> <15539.49195.297507.616138@12-248-41-177.client.attbi.com> <20020410043629.GA6483@panix.com> <15539.50874.831225.914697@12-248-41-177.client.attbi.com> Message-ID: <20020410045956.GA10215@panix.com> On Tue, Apr 09, 2002, Skip Montanaro wrote: > > >> If churning out a binary installer *suitable for a development > >> branch* can be reduced to a few minutes, fine. If not ... > > aahz> Seems to me that for dev releases, there's no need for a "Windows > aahz> installer". A zip file that contains the necessary install tree > aahz> ought to be sufficient. > > I think the contention was implicitly that many people who might be able to > help test development releases couldn't because they lack the necessary > tools to build from source. Sorry, I guess I'm still too Windows-centric in assuming that "install tree" would be generally understood as containing executables. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From skip@pobox.com Wed Apr 10 06:02:21 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 10 Apr 2002 00:02:21 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <20020410044810.GA4932@lilith.my-fqdn.de> References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> <20020410044810.GA4932@lilith.my-fqdn.de> Message-ID: <15539.51037.12990.518662@12-248-41-177.client.attbi.com> Gerhard> But some of these are hard to build and most users don't know Gerhard> there are perfectly good free compilers for Windows. Perhaps this sort of thing could be documented on the Windows page. I presume the Cygwin environment is advanced enough to produce MSVC-compatible EXE files by now. Skip From aahz@pythoncraft.com Wed Apr 10 06:11:38 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 10 Apr 2002 01:11:38 -0400 Subject: [Python-Dev] Developer resources (was Re: Stability and change) In-Reply-To: <006f01c1dfe3$8ce6c9f0$8001a8c0@COMPUTER> References: <15538.32008.560965.639327@grendel.zope.com> <006f01c1dfe3$8ce6c9f0$8001a8c0@COMPUTER> Message-ID: <20020410051138.GA11915@panix.com> On Tue, Apr 09, 2002, Steve Holden wrote: > Fred Drake wrote: >> >> (BTW, is the Starship membership closed now? If so, the starship home >> page should be updated.) > > Well, certainly the starship and (earlier) the PSA were indistinguishable > from /dev/null from the point of view of a relative newcomer (who had paid > for a conference without attending it) wanting a starship account. When was this? There was a while when the Starship admins were mostly off the air, but I believe things have improved. (Or I wouldn't have suggested making Starship a focus of dev efforts.) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From rwgk@cci.lbl.gov Wed Apr 10 06:22:41 2002 From: rwgk@cci.lbl.gov (Ralf W. Grosse-Kunstleve) Date: Tue, 9 Apr 2002 22:22:41 -0700 (PDT) Subject: [Python-Dev] Re: About Pyrex Message-ID: <200204100522.g3A5MfJ08597@boa.lbl.gov> > I had heard of it, but hadn't looked very closely at it. > It looks interesting -- you seem to be approaching the > problem from exactly the opposite direction, in a way. > It's hard to get an idea of how your approach stacks > up against mine, because they're so different. It is not really "my" approach. David Abrahams deserves all the credit. I came to Boost.Python as a user. My contributions are fairly minor in the grand scheme of things. > One thing I can say is that yours stands a better chance > of coping with C++ in its full generality than mine > does at the moment! The most important feature for me is that both the compiled code and the interpreted code look nice and are extensible. The second most important feature is that all you need is Python and C++. There is no third (little) language to learn (as with SWIG). > No, because in that example there's a lot of computation > going on between each result. The inner loop there isn't > the one collecting results, it's the one doing the divisions, > which runs entirely in C. In my world your example is not very typical. The algorithms that I push to the compiled part of the system are usually operations that work on arrays, and almost universally involve custom types (classes). > In general -- if Python code needs to get at the results, > they have to be converted to Python objects somewhere, > somehow. I don't see a way around that. Boost.Python gets around this by making it very easy to expose custom types (C++ classes) to Python, incl. methods, dictionary, pickle support etc. You may view Boost.Python as a high-performance extension class factory. With the new Python 2.2 type system the Boost.Python extension classes (in the ongoing rewrite) are indistinguishable from regular built-in types. In future versions of Python this will also be true for the native "new style" Python classes. I.e., there is no need to convert C++ types from and to traditional Python built-in types. Essentially, Boost.Python just attaches a reference count. (David, please correct me if this view is not accurate.) Ralf From haering_python@gmx.de Wed Apr 10 06:05:56 2002 From: haering_python@gmx.de (Gerhard =?iso-8859-15?Q?H=E4ring?=) Date: Wed, 10 Apr 2002 07:05:56 +0200 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15539.51037.12990.518662@12-248-41-177.client.attbi.com> References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> <20020410044810.GA4932@lilith.my-fqdn.de> <15539.51037.12990.518662@12-248-41-177.client.attbi.com> Message-ID: <20020410050556.GA5094@lilith.my-fqdn.de> * Skip Montanaro [2002-04-10 00:02 -0500]: > > Gerhard> But some of these are hard to build and most users don't know > Gerhard> there are perfectly good free compilers for Windows. > > Perhaps this sort of thing could be documented on the Windows page. I > presume the Cygwin environment is advanced enough to produce MSVC-compatible > EXE files by now. No, it isn't. I was working one some patches to make this a possibility, but because of a stupid mistake, I lost them. I plan to work on this subject again, though. What is needed are mostly changes to pyconfig.h.in, configure.in and Makefile.pre.in. And a little distutils tweaking. But it sure won't work out of the box now. Gerhard -- This sig powered by Python! Außentemperatur in München: 1.6 °C Wind: 1.5 m/s From tim_one@email.msn.com Wed Apr 10 06:53:41 2002 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 10 Apr 2002 01:53:41 -0400 Subject: [Python-Dev] Developer resources (was Re: Stability and change) In-Reply-To: <20020410041923.GA5501@panix.com> Message-ID: >>>> The problem with starship is that there are no criteria to admit new >>>> users. >>> There wasn't any difficulty for me in getting a Starship account a bit >>> more than a year ago. >> See? That proves my theory. :-) > And how is that "a problem"? Are you worried about hordes of people > descending on Starship? Guido was making a joke. I laughed. You should too. From skip@pobox.com Wed Apr 10 07:04:40 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 10 Apr 2002 01:04:40 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <20020410050556.GA5094@lilith.my-fqdn.de> References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> <20020410044810.GA4932@lilith.my-fqdn.de> <15539.51037.12990.518662@12-248-41-177.client.attbi.com> <20020410050556.GA5094@lilith.my-fqdn.de> Message-ID: <15539.54776.352252.534872@12-248-41-177.client.attbi.com> Gerhard> But some of these are hard to build and most users don't know Gerhard> there are perfectly good free compilers for Windows. >> Perhaps this sort of thing could be documented on the Windows page. >> I presume the Cygwin environment is advanced enough to produce >> MSVC-compatible EXE files by now. Gerhard> What is needed are mostly changes to pyconfig.h.in, Gerhard> configure.in and Makefile.pre.in. And a little distutils Gerhard> tweaking. But it sure won't work out of the box now. Thanks for the info. I was referring in general to Cygwin -> MSVC capability. I presume that since you are working on Python-specific patches the general capability is there, just that it's not there quite yet for Python. Skip From David Abrahams" Message-ID: <01d301c1e055$2928b460$6801a8c0@boostconsulting.com> ----- Original Message ----- From: "Ralf W. Grosse-Kunstleve" > The most important feature for me is that both the compiled code and > the interpreted code look nice and are extensible. > The second most important feature is that all you need is Python > and C++. There is no third (little) language to learn (as with > SWIG). As libraries start to take more advantage of template metaprogramming techniques, their interface starts to approach a language definition. How different, really, is the following from being an IDL? BOOST_PYTHON_MODULE_INIT(back_reference_ext) { module("back_reference_ext") .def("copy_Y", copy_Y, return_value_policy()) .def("copy_Z", copy_Z, return_value_policy()) .def("x_instances", &X::count) .add( class_("Y") .def_init(args()) .def("value", &Y::value) .def("set", &Y::set) ) .add( class_ >("Z") .def_init(args()) .def("value", &Z::value) .def("set", &Z::set) ) ; } The real advantage to using the C++ compiler as opposed to intermediate language is that we can be sure it is up to the job of dealing with any C/C++ language constructs we throw at it. -Dave From tim_one@email.msn.com Wed Apr 10 07:08:58 2002 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 10 Apr 2002 02:08:58 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15539.49195.297507.616138@12-248-41-177.client.attbi.com> Message-ID: [Skip Montanaro] > ... > The only way a significant change in the development model is > going to work is if the effort necessary to generate micro releases drops > drastically. I'm still vague on what frequent microreleases are thought to buy us, but I'll skip that here. > If churning out a binary installer *suitable for a development branch* > can be reduced to a few minutes, fine. If not, then I suspect either > someone else will have to take that task over from Tim or Windows > installers won't get cut for development point releases. Building a Windows installer goes fast. It's usually only the testing that takes time. It's a GUI, not a makefile, and has lots of buttons and options, sets up lots of stuff specific to Windows, stuff that needs to be done differently under the covers across different flavors of Windows, and it needs to be tested across different flavors of Windows under user accounts with different privilege levels, and on and on. Windows is a friggin' mess this way. The other thing that bites is that *because* I do so much testing, I'm the one who finds the last-second bugs (not mentioning pydoc by name ). When that happens, the whole world grinds to a halt in emergency mode until it's fixed. I don't know whether the SF file page is still intact, but if it is you can see that we get *most* prerelease downloads from the Windows builds, whether it's the first alpha or the final release candidate. OK, the page is still partly useful: we got 1700+ downloads of the Windows 2.2.1c2, and about a third of that for the Unix tarball. It's hard to know exactly what to make of that, though, as Windows users seem to download anything with an .exe extension without knowing why <0.5 wink -- but thousands of people routinely downloaded the Windows debug-build zip file when I used to ship that too, and I've counted the number of complaints since I stopped shipping it on one hand>. If we want to release an untested installer, it will go much faster. It's not in my nature to do so, but I can be paid to be unprofessional . From tim_one@email.msn.com Wed Apr 10 07:11:51 2002 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 10 Apr 2002 02:11:51 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <20020410043629.GA6483@panix.com> Message-ID: > Seems to me that for dev releases, there's no need for a "Windows > installer". A zip file that contains the necessary install tree ought > to be sufficient. The Windows install tree is a radical rearrangement of the Windows build tree: building an installer and running it is the fastest way to build an install tree. Building a zip file on top of that would actually take more time (BTW, the Windows installer .exe *is* a kind of standard zip file: you can open it directly with WinZip and browse its contents). From martin@v.loewis.de Wed Apr 10 07:22:02 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 10 Apr 2002 08:22:02 +0200 Subject: [Python-Dev] test_mpz skipped? In-Reply-To: <0GUC007KO0J5VT@mtaout02.icomcast.net> References: <0GUC007KO0J5VT@mtaout02.icomcast.net> Message-ID: Jeremy Hylton writes: > > test test_mpz skipped -- No module named mpz > > 1 test skipped: > > test_mpz > > 1 skip unexpected on linux2: > > test_mpz > > Is this just a question of adding test_mpz to the list of expected > skips on Linux? Or is there some problem with my build? No, and no. The notion of "per-platform expected skipped tests" is inherently broken for targets where you usually build from source (i.e. Unix): It depends much on the set of libraries that you have available. It would be much better if somebody rewrote this aspect to analyse the expected failures more precisely. E.g. if module mpz was not built, test_mpz should be skipped; if it was built but fails to import, that would be unexpected. Regards, Martin From mwh@python.net Wed Apr 10 09:22:52 2002 From: mwh@python.net (Michael Hudson) Date: 10 Apr 2002 09:22:52 +0100 Subject: [Python-Dev] 2.2.1 release mechanics In-Reply-To: Jack Jansen's message of "Tue, 9 Apr 2002 23:39:42 +0200" References: <4D430248-4C02-11D6-BA9A-003065517236@oratrix.com> Message-ID: <2mn0wcezdv.fsf@starship.python.net> Jack Jansen writes: > On zondag, maart 31, 2002, at 10:39 , Michael Hudson wrote: > > April 10 ~1200 GMT: release ends > > > > By now, F, T & J have done their bits, uploaded > > files to creosote and sf (or pointed me to where I > > can get them), etc. > > Michael, > my stuff is done. I've announced it on the pythonmac-sig and > given people there 12 hours to find any showstopper bugs, so > unless I ask you not to go ahead within twelve hours it's all > yours (I understand that we're not doing sf releases anymore, > and I don't have access to creosote). The files are in > http://www.cwi.nl/ftp/jack/python/mac, you want MacPython221full > (both .hqx and .bin) and MacPython221src (both .hqx and .sit). > You may also want MacPython221MacOS81patch (.hqx and .sit) which > makes this release run on really old MacOS versions. OK. > If you are going to put a blurb in your message announcing 2.2.1 > there are two fixes in it that are very important to Mac users: > - This release finally works on Mac OS X multiprocessors > - The IDE is now fully functional on Mac OS X. OK. The draft blurb is here: http://starhip.python.net/crew/mwh/anndraft.txt Cheers, M. -- I've even been known to get Marmite *near* my mouth -- but never actually in it yet. Vegamite is right out. UnicodeError: ASCII unpalatable error: vegamite found, ham expected -- Tim Peters, comp.lang.python From mal@lemburg.com Wed Apr 10 09:28:13 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 10 Apr 2002 10:28:13 +0200 Subject: [Python-Dev] Unittests References: <3CB3093C.B7A22727@metaslash.com> <20020409210943.GB16531@panix.com> Message-ID: <3CB3F79D.A143D1C2@lemburg.com> Aahz wrote: > > On Tue, Apr 09, 2002, Neal Norwitz wrote: > > > > whrandom.py > > This is a deprecated module, no? I certainly hope not. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From mal@lemburg.com Wed Apr 10 09:34:10 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 10 Apr 2002 10:34:10 +0200 Subject: [Python-Dev] Re: Regression in unicodestr.encode()? References: Message-ID: <3CB3F902.49E3FE76@lemburg.com> It's probably a bug. Barry, please file a bug and assign it to me. Thanks, -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From mal@lemburg.com Wed Apr 10 09:36:07 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 10 Apr 2002 10:36:07 +0200 Subject: [Python-Dev] Regression in unicodestr.encode()? References: <15539.16595.651881.216153@yyz.zope.com> <15539.39981.221392.757470@anthem.wooz.org> Message-ID: <3CB3F977.38A867A5@lemburg.com> "Barry A. Warsaw" wrote: > > I've submitted a SF bug on this and assigned it to MAL. > > http://sourceforge.net/tracker/index.php?func=detail&aid=541828&group_id=5470&atid=105470 Thanks. Forget my last message. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From mwh@python.net Wed Apr 10 09:41:34 2002 From: mwh@python.net (Michael Hudson) Date: 10 Apr 2002 09:41:34 +0100 Subject: [Python-Dev] Regression in unicodestr.encode()? In-Reply-To: barry@zope.com's message of "Tue, 9 Apr 2002 15:28:19 -0400" References: <15539.16595.651881.216153@yyz.zope.com> Message-ID: <2mpu182bep.fsf@starship.python.net> barry@zope.com (Barry A. Warsaw) writes: > Is the utf-8 encoder in cvs broken? I seem to remember that test_unicode failed the last time I built a wide unicode build from the trunk. Someone else can find the time to try this again, or I'll do it in a few days... Cheers, M. -- Lisp does badly because we refuse to lie. When people ask us if we can solve insoluble problems we say that we can't, and because they expect us to lie to them, they find some other language where the truth is less respected. -- Tim Bradshaw, comp.lang.lisp From mal@lemburg.com Wed Apr 10 10:12:12 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 10 Apr 2002 11:12:12 +0200 Subject: [Python-Dev] Re: Deprecation process (random vs whrandom) References: <3CB39D8E.C5E837CE@metaslash.com> <200204100223.g3A2Nvp19644@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CB401EC.AF323D3C@lemburg.com> Guido van Rossum wrote: > > > I added warnings to statcache & xmllib which are deprecated in the docs. > > Any others need to be done? Should statcache be removed from test_sundry? > > Should any modules be moved into lib-old? posixfile? > > A good exercise would be to do a 1-minute review of every module in > the library and decide for yourself if you'd deprecate it if you were > BDFL. I'll gladly review the list. I must say that I'm not very thrilled about the PythonLabs strategy to go about and deprecate lots of modules which have been in use for years, documented in many Python books, used as examples in tutorials, slides, newsgroup postings, etc. just for the fun of it. What has happened to the conservative strategy of achieving backwards compatibility as highest priority ? Why can't whrandom simply redirect to random ? (without deprecation warnings which confuse newbies and bother long-time users) Why do we have to deprecate code which was replaced with a compatible API in a differently named module ? (it can't be the few bytes we save in the distro...) I don't see the point of carelessly breaking stuff for cosmetic reasons. I'm not at all objected to replacing older technology with new one, but I am worried about the way this is done. To me, all this generates is FUD, but unfortunately this time for a very real reason... you can no longer be sure that your script which you wrote two years ago will still run with the Python version released within the next year and that's seriously bad publicity for Python which has had a long track record of being a very stable, very neat platform for writing code that you can not only *read* two years after having written it, but also run without change. Hmm, I should add a smiley somehwere here... :-) <-- there it is ;-) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From greg@cosc.canterbury.ac.nz Wed Apr 10 10:30:06 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Wed, 10 Apr 2002 21:30:06 +1200 (NZST) Subject: [Python-Dev] Re: About Pyrex In-Reply-To: <200204100522.g3A5MfJ08597@boa.lbl.gov> Message-ID: <200204100930.g3A9U6903943@oma.cosc.canterbury.ac.nz> > In my world your example is not very typical. The algorithms that I > push to the compiled part of the system are usually operations that > work on arrays, and almost universally involve custom types > (classes). I know it's probably not typical, but it's the best I could come up with on the spur of the moment with the amount of Pyrex that currently works. I can think of analogous things that could arise in real life, though, e.g. if you're wrapping an FFT algorithm or linear algebra package or something, and you want to be able to pass your data in and out as lists of floats without having to put them in a special data structure first. > Boost.Python gets around this by making it very easy to expose custom > types (C++ classes) to Python, incl. methods, dictionary, pickle > support etc. You may view Boost.Python as a high-performance extension > class factory. Yes, I can see that. Something similar is planned for Pyrex, too. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From mwh@python.net Wed Apr 10 10:49:49 2002 From: mwh@python.net (Michael Hudson) Date: 10 Apr 2002 10:49:49 +0100 Subject: [Python-Dev] Re: Stability and change In-Reply-To: David Ascher's message of "Tue, 09 Apr 2002 09:42:59 -0700" References: <3CB31A13.EF1BB615@activestate.com> Message-ID: <2mpu17lw76.fsf@starship.python.net> David Ascher writes: > On the topic of unit tests, I wonder if it makes sense to consider a new > test directory outside of Lib/. As the test suite is beefed up > (something which I think is a great idea), the disk space requirement > for the test suite shouldn't be a factor -- the more non-overlapping > tests the better, period -- but if it's part of the library tree, space > _should_ be a concern because that stuff gets shipped to the millions of > users. > > In other words, I'd like to have a test suite outside of what gets > installed. The reason for not moving all of it out is that there is > value in being able to run a post-install test suite, but I don't think > it needs to be all of the tests. > > Reactions? Not unless there are tests which require significant diskspace. Lib/test compresses to about 500k now, in a 6.5 meg tarball. Cheers, M. -- at any rate, I'm satisfied that not only do they know which end of the pointy thing to hold, but where to poke it for maximum effect. -- Eric The Read, asr, on google.com From mwh@python.net Wed Apr 10 10:58:53 2002 From: mwh@python.net (Michael Hudson) Date: 10 Apr 2002 10:58:53 +0100 Subject: [Python-Dev] 2.3 goals In-Reply-To: Tim Peters's message of "Tue, 09 Apr 2002 16:12:53 -0400" References: Message-ID: <2mn0wblvs2.fsf@starship.python.net> Tim Peters writes: > [Andrew Kuchling] > > Oh, and another possible 2.3 thing: is it time to drop the PCRE code? > > Whatever /F says. Over and over we've gotten reports that some naively > written regexps that work OK (albeit slowly) under pre "blow the stack" > under sre. The suggested workaround has been to "import pre as re". So we > break working code if the pre engine goes away before /F thinks sre is as > forgiving in these nasty cases. Stackless sre! Where's CT when you need him? Cheers, M. -- ... but I'd rather not reinvent the wheel if I don't have to. On the other hand, if the currently instantiated version of the wheel consists of a square rock covered with moss, I might as well just start fresh. -- Roy Smith, comp.lang.python From mwh@python.org Wed Apr 10 11:47:24 2002 From: mwh@python.org (Michael Hudson) Date: Wed, 10 Apr 2002 11:47:24 +0100 (BST) Subject: [Python-Dev] RELEASED: Python 2.2.1 Message-ID: We've released a bugfix release of Python, 2.2.1! This release should be fully compatible with Python 2.2. Everyone using 2.2 is recommended to upgrade to 2.2.1. On the 2.2.1 pages http://www.python.org/2.2.1/ you can find more information, binaries for Windows, Redhat 7.2 & 6.2, Mandrake 8.2 and MacOS 8, 9 & X, as well as the usual tarball. Thanks are due to those who provided the binaries and to all who tested the prerelases during the 2.2.1 release process! One nasty bug (that affected Zope in obscure circumstances) and a few less serious ones have been fixed since the second release candidate. Two fixes may be of special interest if you use MacOS X: - This release finally works on Mac OS X multiprocessors - The IDE is now fully functional on Mac OS X. This being a bugfix release, there are no exciting new features -- we just fixed a lot of bugs. For a moderately complete list, please see: http://www.python.org/2.2.1/NEWS.txt Cheers, The Python development team. From hernan@orgmf.com.ar Wed Apr 10 12:49:42 2002 From: hernan@orgmf.com.ar (Hernan Martinez Foffani) Date: Wed, 10 Apr 2002 13:49:42 +0200 Subject: [Python-Dev] Unittest list Message-ID: From: Neal Norwitz > I pulled it from where the sun don't shine. :-) > I did a bunch of ls/grep/echo/... commands. > Then post processed manually. > > It's not maintained anywhere. It would be great if there was > a place to maintain/update it. Suggestions? > > I have another script which builds python with test-coverage > and will tell you each line/function that doesn't get executed > in case you get really bored. :-) I'm interested in those scripts. Are they available for the public? > > There's only about 200 functions in Objects/*.c. That "200", is the aprox number of functions that don't have a corresponding unittest? By the way, is there a simmilar package that do the same thing (tell which line/function that doesn't get executed) for pure python programs? I mean, something that can run with the standard Python distrib. Regards, -Hernan From sholden@holdenweb.com Wed Apr 10 13:34:41 2002 From: sholden@holdenweb.com (Steve Holden) Date: Wed, 10 Apr 2002 08:34:41 -0400 Subject: [Python-Dev] Developer resources (was Re: Stability and change) References: <15538.32008.560965.639327@grendel.zope.com> <006f01c1dfe3$8ce6c9f0$8001a8c0@COMPUTER> <20020410051138.GA11915@panix.com> Message-ID: <002e01c1e08c$16fae830$8001a8c0@COMPUTER> Aahz wr ote: > On Tue, Apr 09, 2002, Steve Holden wrote: > > Fred Drake wrote: > >> > >> (BTW, is the Starship membership closed now? If so, the starship home > >> page should be updated.) > > > > Well, certainly the starship and (earlier) the PSA were indistinguishable > > from /dev/null from the point of view of a relative newcomer (who had paid > > for a conference without attending it) wanting a starship account. > > When was this? There was a while when the Starship admins were mostly > off the air, but I believe things have improved. (Or I wouldn't have > suggested making Starship a focus of dev efforts.) Aeons ago. Well, over two years, anyway -- the reason I didn't get to the conference was a confluence of machine crash and too much snow (this was the Rosslyn event). The reception was so underwhelming I eventually decided not to bother. regards Steve From skip@pobox.com Wed Apr 10 13:46:51 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 10 Apr 2002 07:46:51 -0500 Subject: [Python-Dev] Unittests In-Reply-To: <3CB3F79D.A143D1C2@lemburg.com> References: <3CB3093C.B7A22727@metaslash.com> <20020409210943.GB16531@panix.com> <3CB3F79D.A143D1C2@lemburg.com> Message-ID: <15540.13371.699769.859830@12-248-41-177.client.attbi.com> >> > whrandom.py >> This is a deprecated module, no? mal> I certainly hope not. So you're the culprit! :-) S From jeremy@zope.com Wed Apr 10 13:57:30 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Wed, 10 Apr 2002 08:57:30 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15539.49195.297507.616138@12-248-41-177.client.attbi.com> References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> <15539.49195.297507.616138@12-248-41-177.client.attbi.com> Message-ID: <15540.14010.181095.298392@slothrop.zope.com> >>>>> "SM" == Skip Montanaro writes: SM> Oh well. Guido already complains (rightfully so, I believe) SM> that there is very little community input on the existing SM> alpha/beta/rc1/rc2 process. As he put it earlier today, people SM> who check out 2.2final think they are early adopters. If most SM> people don't test out early releases now, Tim's been wasting his SM> time generating all those installers. Here you say: Few people download and test development releases. SM> The only way a significant change in the development model is SM> going to work is if the effort necessary to generate micro SM> releases drops drastically. Here you say: Let's do more development releases. Why? Jeremy From pobrien@orbtech.com Wed Apr 10 14:20:40 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Wed, 10 Apr 2002 08:20:40 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <20020410044810.GA4932@lilith.my-fqdn.de> Message-ID: [Gerhard Häring] > > I doubt that there's a lot of input from Windows users for alphas/betas > for major releases. The reason is that if they needed to test their > software, they'd also need new builds of the third-party packages they > use on Windows. But some of these are hard to build and most users don't > know there are perfectly good free compilers for Windows. Let's say that I fall into this camp and would like to be able to compile Python on Windows 98. What good free compiler do you recommend, what are the minimum steps I need to follow to install it, and what are the minimum steps I need to follow to compile the latest Python from CVS? (I already know how to use CVS fairly well.) If the only real barrier is a lack of information, let's fix that problem first. Then we can see whether any Windows users are serious about testing alphas and betas. This is the only way we'll make any progress. --- Patrick K. O'Brien Orbtech From guido@python.org Wed Apr 10 14:33:35 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 10 Apr 2002 09:33:35 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Tue, 09 Apr 2002 20:32:10 PDT." <3CB3B23A.2277F6CE@prescod.net> References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> <15537.43727.369744.356521@grendel.zope.com> <3CB2AE2F.2DAF6373@prescod.net> <200204100109.g3A19s119435@pcp742651pcs.reston01.va.comcast.net> <3CB3B23A.2277F6CE@prescod.net> Message-ID: <200204101333.g3ADXZw22133@pcp742651pcs.reston01.va.comcast.net> > Guido van Rossum wrote: > > > The cure for the pain is the same as for any other third-party > > > module: tie newer releases to newer Python versions. > > > > But that's exactly what's unacceptable in theis case: newer versions > > of PyXML provide new PyXML features, and Python users who wish to use > > the new PyXML features should *not* have to upgrade to the latest and > > greatest Python. > > I still don't see how this is any different than any other Python > module. It's probably painful keeping NumPy compatible with old versions > of Python also. How is PyXML unique in this? > > Paul Prescod I didn't say that. NumPy has exactly the same issues. You proposed to tie newer versions of 3rd party modules to newer Python releases. I think it's wise to stay at least 1 or 2 releases behind. You don't want to lose your users because you forced them into a bad upgrade of the rest of their Python installation. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Apr 10 14:34:16 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 10 Apr 2002 09:34:16 -0400 Subject: [Python-Dev] NEWS consolidation In-Reply-To: Your message of "Tue, 09 Apr 2002 20:36:11 PDT." <3CB3B32B.677A3C1D@ActiveState.com> References: <200204081528.g38FSi112614@mbuna.arbhome.com.au> <20020408163903.GA2216@crystal.mems-exchange.org> <200204081724.g38HOOB13065@pcp742651pcs.reston01.va.comcast.net> <15538.194.350128.498326@12-248-41-177.client.attbi.com> <200204082310.g38NAUD14877@mercur.uphs.upenn.edu> <200204090019.g390J3C15946@pcp742651pcs.reston01.va.comcast.net> <200204091420.g39EKPF16144@mercur.uphs.upenn.edu> <20020409144717.GA7946@crystal.mems-exchange.org> <20020409230554.GC10503@panix.com> <200204092329.g39NT8o18942@pcp742651pcs.reston01.va.comcast.net> <15539.35323.583803.887661@grendel.zope.com> <200204100053.g3A0rtk19316@pcp742651pcs.reston01.va.comcast.net> <3CB3B32B.677A3C1D@ActiveState.com> Message-ID: <200204101334.g3ADYGu22144@pcp742651pcs.reston01.va.comcast.net> > > > XML is never high on your list of suggestions. On the other hand, > > > it's exactly appropriate for this task. > > > > Not if you ever want me to supply a news item again. XML is > > unreadable and untypable. > > Think of XML as a database format. The input could be in a formalized > version of the current NEWS format, and the output could be whatever > people wanted to write. It's not a bad database format for textual > data. Better than pickles, IMO. As long as I don't have to type the XML, I'm fine with it. --Guido van Rossum (home page: http://www.python.org/~guido/) From pinard@iro.umontreal.ca Wed Apr 10 15:04:11 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 10 Apr 2002 10:04:11 -0400 Subject: [Python-Dev] Re: Regression in unicodestr.encode()? In-Reply-To: <20020409214611.B1356@unpythonic.dhs.org> References: <15539.16595.651881.216153@yyz.zope.com> <200204092257.g39MvRi18636@pcp742651pcs.reston01.va.comcast.net> <200204100050.g3A0oN319241@pcp742651pcs.reston01.va.comcast.net> <20020409214611.B1356@unpythonic.dhs.org> Message-ID: [jepler@unpythonic.dhs.org] > Why Python refuses to do it this way: > for security reasons, the UTF-8 codec gives you an "illegal encoding" > error in this case. > [...] I'm terribly glad that Python has gotten this detail right. I'm also glad that Python did it right, not at all because of security reasons (these are debatable -- the trend is to see security holes everywhere in these days), but for better conformance with Unicode specifications. Python being 8-bit clean, it is less a problem with it than with languages much relying on NUL terminated C strings. I hope that Python will stick to its current UTF-8 behaviour, even if C extension writers were applying some pressure for a change. -- François Pinard http://www.iro.umontreal.ca/~pinard From skip@pobox.com Wed Apr 10 15:16:24 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 10 Apr 2002 09:16:24 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15540.14010.181095.298392@slothrop.zope.com> References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> <15539.49195.297507.616138@12-248-41-177.client.attbi.com> <15540.14010.181095.298392@slothrop.zope.com> Message-ID: <15540.18744.580893.827157@12-248-41-177.client.attbi.com> SM> If most people don't test out early releases now, Tim's been wasting SM> his time generating all those installers. Jeremy> Here you say: Few people download and test development releases. SM> The only way a significant change in the development model is going SM> to work is if the effort necessary to generate micro releases drops SM> drastically. Jeremy> Here you say: Let's do more development releases. Jeremy> Why? I think I need to write a PEP. I suppose I would have by now, but it seems that most of my mails on this subject are falling on deaf ears. There is perception and there is reality. I believe the reality is that Python is pretty stable. There seems to be a perception outside of the python-dev community that it is not. In large measure this thread has been more about what to do about perceived instability than actual instability. Way back on April 8th, I sent this note to python-dev: The problem with simply dealing off the top of the CVS deck is that it provides no way to conclusively state that the current CVS head is either more functionally complete or more bug-free than an earlier CVS head. Hopefully releases will be must smaller efforts than they currently are. Pulling away from SF for releases will certainly help. If a micro release on the development branch can asymptotically approach * create a baseline named devel-2.N.x * drop a tarfile onto www.python.org * send out an email I think you could probably justify making releases on that branch every month or so. Releases on the stable branch would probably occur less frequently (only when a sufficient number of bug fixes had accumulated or a particularly serious bug fix was applied to the branch) and be a bit more involved (maybe involving creation of a Windows installer). Above and beyond the notion of complexity or technical superiority of different development models, I think you need to realize a positive PR advantage from a change in development models. If you could produce a table like Version Bugs fixed ------- ---------- 2.4.1 47 2.4.2 32 2.4.3 12 2.4.4 3 you would go a long way toward arguing that "stable" branches are really getting more stable. Similarly, such a table on the development branch might help developers decide if the development branch is stable enough to fork. You need to make measurements that demonstrate to people that your code base is getting more stable, if only for PR purposes. Associating those metrics with specific release points would, I think, provide more confidence in those numbers. It's cheap to add a new tag to CVS. It's not much more expensive to dump a tarball associated with that tag. That's all I'm really suggesting you do along the development branch. Hell, you won't even need a release manager. The PLabs gang could announce they are going to take a snapshot Friday, ask for a freeze at 10AM EST, have their weekly staff meeting, create the tag at noon, and have a tarball on the website by 12:30, including updating a table on the as-yet-nonexistent /dev/downloads page. Even if not a single soul downloads and builds a particular snapshot, creating such snapshots helps because they provide you with static points in the CVS repository from which to make measurements. They are, in effect, the X axis values of a plot of whatever other measurements you make on the code base (number of modules, lines of code changed, etc). "You" here means "you, the overall community". I'm not suggesting that PythonLabs take on the extra burden of making a bug of measurements. Just provide the tickmarks for the graph. Skip From aahz@pythoncraft.com Wed Apr 10 15:19:34 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 10 Apr 2002 10:19:34 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: <20020410043629.GA6483@panix.com> Message-ID: <20020410141934.GB10792@panix.com> On Wed, Apr 10, 2002, Tim Peters wrote: > >> Seems to me that for dev releases, there's no need for a "Windows >> installer". A zip file that contains the necessary install tree ought >> to be sufficient. > > The Windows install tree is a radical rearrangement of the Windows build > tree: building an installer and running it is the fastest way to build an > install tree. Building a zip file on top of that would actually take more > time (BTW, the Windows installer .exe *is* a kind of standard zip file: you > can open it directly with WinZip and browse its contents). Ah. Okay, is shipping a zip of the build tree (plus a couple of MSVCRT libraries) sufficient to run Python? Would that be any less effort? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From jacob@strakt.com Wed Apr 10 15:22:47 2002 From: jacob@strakt.com (=?ISO-8859-1?Q?Jacob_Hall=E9n?=) Date: Wed, 10 Apr 2002 16:22:47 +0200 Subject: [Python-Dev] Multiple revisions [was: Re: Stability and change] Message-ID: <3CB44AB7.8010600@strakt.com> >>/ You can significantly reduce the time needed to apply patches to seve= ral />>/ versions of your software by using a smart revision control system. />>/=20 />>/ I recently discovered "arch", which has been designed to handle mult= iple />>/ parallell versions. It is a marvel, and it comes with really good />>/ descriptions of how to handle the problem in practice. I know that />>/ Linus Torvalds is considering it for the Linux kernel. />>/=20 />>/ http://regexps.com/ /> > If I had to start over, I'd be happy to consider arch (and Perforce, > and Bitkeeper, and, oh, why not, Subversion). As it is, this would > mean ripping out the foundation from our development process -- > everything we do is tied to CVS on SF. So, unless arch runs on top > of an existing CVS setup, too bad. Perforce, Bitkeeper and Subversion are all "better-breed-CVS" solutions, while I consider arch to be a "better-breed-revision-control-system". Unfortunately it currently doesn't have an import mechanism that will kee= p all your CVS history, so I understand that it is not an option at this time. However, it is a young project, having its first alpha release on Jan 16 of this year. At the current development pace, we may see the CVS import functionality very soon. I have sent a question about this to Tom Lord. Jacob Hall=E9n From SBrunning@trisystems.co.uk Wed Apr 10 15:23:16 2002 From: SBrunning@trisystems.co.uk (Simon Brunning) Date: Wed, 10 Apr 2002 15:23:16 +0100 Subject: [Python-Dev] Re: Stability and change Message-ID: <31575A892FF6D1118F5800600846864DCBCF75@intrepid> > From: Skip Montanaro [SMTP:skip@pobox.com] > There is perception and there is reality. I believe the reality is that > Python is pretty stable. There seems to be a perception outside of the > python-dev community that it is not. In large measure this thread has > been > more about what to do about perceived instability than actual instability. There is a perception *by a vocal minority* of those outside of python-dev that Python isn't stable. Cheers, Simon Brunning TriSystems Ltd. sbrunning@trisystems.co.uk ----------------------------------------------------------------------- The information in this email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this email by anyone else is unauthorised. If you are not the intended recipient, any disclosure, copying, distribution, or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. TriSystems Ltd. cannot accept liability for statements made which are clearly the senders own. From aahz@pythoncraft.com Wed Apr 10 15:28:24 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 10 Apr 2002 10:28:24 -0400 Subject: [Python-Dev] Unittests In-Reply-To: <3CB3F79D.A143D1C2@lemburg.com> References: <3CB3093C.B7A22727@metaslash.com> <20020409210943.GB16531@panix.com> <3CB3F79D.A143D1C2@lemburg.com> Message-ID: <20020410142824.GC10792@panix.com> On Wed, Apr 10, 2002, M.-A. Lemburg wrote: > Aahz wrote: >> On Tue, Apr 09, 2002, Neal Norwitz wrote: >>> >>> whrandom.py >> >> This is a deprecated module, no? > > I certainly hope not. Well, you hope wrong. ;-) Argue with the Timbot. Actually, I think the issue of whether to deprecate whrandom is a good concrete point to wrap the stability discussion around. On the one hand, it *is* a no-longer maintained module that has been superseded by another. random.py was certainly available as far back as 1.5.2 (just triple-checked), and Tim at least was constantly pushing people to use random instead of whrandom. OTOH, I was one of the people who used whrandom back then because it was simpler than random (don't remember why ;-). If I were distributing code, I can see being upset that whrandom were being deleted. My suggestion is that whrandom get a warning in 2.3 and 2.4, then deleted in the following version. Note to Tim: you haven't followed PEP 4, so you can't really say that whrandom is deprecated yet. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From aahz@pythoncraft.com Wed Apr 10 15:32:42 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 10 Apr 2002 10:32:42 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15540.14010.181095.298392@slothrop.zope.com> References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> <15539.49195.297507.616138@12-248-41-177.client.attbi.com> <15540.14010.181095.298392@slothrop.zope.com> Message-ID: <20020410143241.GD10792@panix.com> On Wed, Apr 10, 2002, Jeremy Hylton wrote: > >>>>> "SM" == Skip Montanaro writes: > > SM> Oh well. Guido already complains (rightfully so, I believe) > SM> that there is very little community input on the existing > SM> alpha/beta/rc1/rc2 process. As he put it earlier today, people > SM> who check out 2.2final think they are early adopters. If most > SM> people don't test out early releases now, Tim's been wasting his > SM> time generating all those installers. > > Here you say: Few people download and test development releases. Skip's already answered with half of my point. The other half that's critical to this is making it easy and comfortable to try out dev releases and promoting this through the community. We want to encourage people to do this, and in my time on python-dev (plus more time on c.l.py), I have seen little attempt to directly address this. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From skip@pobox.com Wed Apr 10 15:36:03 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 10 Apr 2002 09:36:03 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <31575A892FF6D1118F5800600846864DCBCF75@intrepid> References: <31575A892FF6D1118F5800600846864DCBCF75@intrepid> Message-ID: <15540.19923.569937.419815@12-248-41-177.client.attbi.com> >> There is perception and there is reality. I believe the reality is >> that Python is pretty stable. There seems to be a perception outside >> of the python-dev community that it is not. In large measure this >> thread has been more about what to do about perceived instability >> than actual instability. Simon> There is a perception *by a vocal minority* of those outside of Simon> python-dev that Python isn't stable. Agreed. We don't know how big that vocal minority is, but: * Juding by Guido's response to Andy Robinson's note yesterday, I'd say that at least part of the vocal minority is reasonably well respected. * RedHat's insistence that they stick with 1.5.2 for the time being for what appear to be valid business reasons also supports the Garths in the crowd who don't understand those reasons and take them out of context. That suggests that minority or not, their collective voice carries some weight. Skip From hernan@orgmf.com.ar Wed Apr 10 15:41:36 2002 From: hernan@orgmf.com.ar (Hernan Martinez Foffani) Date: Wed, 10 Apr 2002 16:41:36 +0200 Subject: [Python-Dev] RE: Unittest list In-Reply-To: Message-ID: I wrote: > ... > I'm interested in those scripts. Are they available for the > public? > ... etc ... I guess that it was my first "official" post to the list. Then: I have worked for several years in IT projects for local Telco's as programmer, soft eng, project manager, product manager for a big specialized consulting firm. My last two years I was working as project director for small to medium sized Internet houses. Best regards, -Hernan From tim.one@comcast.net Wed Apr 10 16:02:26 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 10 Apr 2002 11:02:26 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <20020410141934.GB10792@panix.com> Message-ID: [Aahz] > Ah. Okay, is shipping a zip of the build tree (plus a couple of MSVCRT > libraries) sufficient to run Python? Sufficient for whom? Not for the typical Windows user. It's sufficient for a typical developer, but if they want a build tree they can build Python themself from CVS. > Would that be any less effort? If developers built their own Python from CVS, that would certainly be less effort for me . I'll note that some small number of people have said they could be more help on Windows if we had a makefile for a different Windows compiler (and I guess they must mean "not Cygwin", else it's already there, thanks to Jason Tishler). From guido@python.org Wed Apr 10 16:12:33 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 10 Apr 2002 11:12:33 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Wed, 10 Apr 2002 10:32:42 EDT." <20020410143241.GD10792@panix.com> References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> <15539.49195.297507.616138@12-248-41-177.client.attbi.com> <15540.14010.181095.298392@slothrop.zope.com> <20020410143241.GD10792@panix.com> Message-ID: <200204101512.g3AFCXd22433@pcp742651pcs.reston01.va.comcast.net> [Aahz] > Skip's already answered with half of my point. The other half that's > critical to this is making it easy and comfortable to try out dev > releases and promoting this through the community. We want to encourage > people to do this, and in my time on python-dev (plus more time on > c.l.py), I have seen little attempt to directly address this. What's a dev release? An alpha or beta, I presume. Under that definition, every single dev release gets promoted in the community with a posting in c.l.py and c.l.py.announce, a webpage, a Windows installer, and so on. What more do we need? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Apr 10 16:13:52 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 10 Apr 2002 11:13:52 -0400 Subject: [Python-Dev] Unittests In-Reply-To: Your message of "Wed, 10 Apr 2002 10:28:24 EDT." <20020410142824.GC10792@panix.com> References: <3CB3093C.B7A22727@metaslash.com> <20020409210943.GB16531@panix.com> <3CB3F79D.A143D1C2@lemburg.com> <20020410142824.GC10792@panix.com> Message-ID: <200204101513.g3AFDqS22448@pcp742651pcs.reston01.va.comcast.net> > My suggestion is that whrandom get a warning in 2.3 and 2.4, then > deleted in the following version. And that's what we're going to do. --Guido van Rossum (home page: http://www.python.org/~guido/) From jacobs@penguin.theopalgroup.com Wed Apr 10 16:14:00 2002 From: jacobs@penguin.theopalgroup.com (Kevin Jacobs) Date: Wed, 10 Apr 2002 11:14:00 -0400 (EDT) Subject: [Python-Dev] Re: Stability and change In-Reply-To: Message-ID: On Wed, 10 Apr 2002, Tim Peters wrote: > I'll note that some small number of people have said they could be more > help on Windows if we had a makefile for a different Windows compiler (and > I guess they must mean "not Cygwin", else it's already there, thanks to > Jason Tishler). On a partially unrelated note, I've had good luck compiling Python (1.5.2) under MingW[*] with minimal fuss. I still use Cygwin as a build environment so that make, configure, etc. all work. This may turn out to be the easiest and cheapest way to get developers going under Win32. If anyone cares, I can see what it takes to get a threaded build of the current Python CVS going. -Kevin [*] MingW is GCC targeted to native Win32 executables, just like Visual Studio would produce. For more information see: http://www.mingw.org/ -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com From aahz@pythoncraft.com Wed Apr 10 16:15:02 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 10 Apr 2002 11:15:02 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: <20020410141934.GB10792@panix.com> Message-ID: <20020410151502.GA23597@panix.com> On Wed, Apr 10, 2002, Tim Peters wrote: > [Aahz] >> Ah. Okay, is shipping a zip of the build tree (plus a couple of MSVCRT >> libraries) sufficient to run Python? > > Sufficient for whom? Not for the typical Windows user. It's sufficient for > a typical developer, but if they want a build tree they can build Python > themself from CVS. Not me; I don't have a Windows C compiler, and it's extremely unlikely that I would until I get a Win2K box. I think that's representative of a large minority in the Python community. When I said "build tree", I meant after compile/link; before that, it's just a source tree. ;-) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From pobrien@orbtech.com Wed Apr 10 16:31:59 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Wed, 10 Apr 2002 10:31:59 -0500 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Message-ID: [Kevin Jacobs] > > On a partially unrelated note, I've had good luck compiling Python (1.5.2) > under MingW[*] with minimal fuss. I still use Cygwin as a build > environment > so that make, configure, etc. all work. This may turn out to be > the easiest > and cheapest way to get developers going under Win32. If anyone > cares, I can > see what it takes to get a threaded build of the current Python CVS going. I care. (And I've got Cygwin installed already) --- Patrick K. O'Brien Orbtech From guido@python.org Wed Apr 10 16:40:21 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 10 Apr 2002 11:40:21 -0400 Subject: [Python-Dev] Multiple revisions [was: Re: Stability and change] In-Reply-To: Your message of "Wed, 10 Apr 2002 16:22:47 +0200." <3CB44AB7.8010600@strakt.com> References: <3CB44AB7.8010600@strakt.com> Message-ID: <200204101540.g3AFeLM22492@pcp742651pcs.reston01.va.comcast.net> > Perforce, Bitkeeper and Subversion are all "better-breed-CVS" solutions, > while I consider arch to be a "better-breed-revision-control-system". You may be right about Perforce or Subversion (though I doubt it), but the arch blurb I found sounds very similar to Bitkeeper. > Unfortunately it currently doesn't have an import mechanism that will keep > all your CVS history, so I understand that it is not an option at this > time. > > However, it is a young project, having its first alpha release on Jan 16 > of this year. At the current development pace, we may see the CVS import > functionality very soon. I have sent a question about this to Tom Lord. I don't think the Python developer community ought to be early adopters for a revision control system. We'll see how arch does in a few years. (BTW, arch is a lousy name. It's impossible to find using Google.) --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas.heller@ion-tof.com Wed Apr 10 16:38:25 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Wed, 10 Apr 2002 17:38:25 +0200 Subject: [Python-Dev] Re: Stability and change References: <20020410141934.GB10792@panix.com> <20020410151502.GA23597@panix.com> Message-ID: <01bd01c1e0a5$c1422dd0$e000a8c0@thomasnotebook> From: "Aahz" > On Wed, Apr 10, 2002, Tim Peters wrote: > > [Aahz] > >> Ah. Okay, is shipping a zip of the build tree (plus a couple of MSVCRT > >> libraries) sufficient to run Python? > > > > Sufficient for whom? Not for the typical Windows user. It's sufficient for > > a typical developer, but if they want a build tree they can build Python > > themself from CVS. > > Not me; I don't have a Windows C compiler, and it's extremely unlikely > that I would until I get a Win2K box. I think that's representative of > a large minority in the Python community. When I said "build tree", I Which minority ;-) Python users on windows? Windows users without compiler? > meant after compile/link; before that, it's just a source tree. ;-) Most windows users won't care if it's a compiled source tree or a windows installer - they only can test their apps if they have there most beloved extensions installed for this new version. win32all and wxPython seem to be most prominent. Until Mark and/or Robin make installers for those extensions available, noone of this 'large minority' will be able to test hist stuff. So, only windows users with a C compiler will test the release - to see if their own extensions do work, and they can also do it compiling the release themselves. Thomas From guido@python.org Wed Apr 10 16:42:07 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 10 Apr 2002 11:42:07 -0400 Subject: [Python-Dev] Re: Regression in unicodestr.encode()? In-Reply-To: Your message of "10 Apr 2002 10:04:11 EDT." References: <15539.16595.651881.216153@yyz.zope.com> <200204092257.g39MvRi18636@pcp742651pcs.reston01.va.comcast.net> <200204100050.g3A0oN319241@pcp742651pcs.reston01.va.comcast.net> <20020409214611.B1356@unpythonic.dhs.org> Message-ID: <200204101542.g3AFg7622512@pcp742651pcs.reston01.va.comcast.net> > Python being 8-bit clean, it is less a problem with it than with languages > much relying on NUL terminated C strings. I hope that Python will stick > to its current UTF-8 behaviour, even if C extension writers were applying > some pressure for a change. Python won't change its story here. (We *will* get to the bottom of Barry's bug, which had nothing to do with this issue.) --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik@pythonware.com Wed Apr 10 16:33:46 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 10 Apr 2002 17:33:46 +0200 Subject: [Python-Dev] Re: Regression in unicodestr.encode()? References: <15539.16595.651881.216153@yyz.zope.com><200204092257.g39MvRi18636@pcp742651pcs.reston01.va.comcast.net><200204100050.g3A0oN319241@pcp742651pcs.reston01.va.comcast.net><20020409214611.B1356@unpythonic.dhs.org> Message-ID: <019801c1e0a6$62b77df0$ced241d5@hagrid> Fran=E7ois Pinard wrote: > I'm also glad that Python did it right, not at all because of security > reasons (these are debatable -- the trend is to see security holes > everywhere in these days) that's because they are everywhere: for yet another horror story, read this: http://www.phrack.com/phrack/58/p58-0x09 From aahz@pythoncraft.com Wed Apr 10 16:21:36 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 10 Apr 2002 11:21:36 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204101512.g3AFCXd22433@pcp742651pcs.reston01.va.comcast.net> References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> <15539.49195.297507.616138@12-248-41-177.client.attbi.com> <15540.14010.181095.298392@slothrop.zope.com> <20020410143241.GD10792@panix.com> <200204101512.g3AFCXd22433@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020410152135.GB23597@panix.com> On Wed, Apr 10, 2002, Guido van Rossum wrote: > [Aahz] >> Skip's already answered with half of my point. The other half that's >> critical to this is making it easy and comfortable to try out dev >> releases and promoting this through the community. We want to encourage >> people to do this, and in my time on python-dev (plus more time on >> c.l.py), I have seen little attempt to directly address this. > > What's a dev release? An alpha or beta, I presume. Under that > definition, every single dev release gets promoted in the community > with a posting in c.l.py and c.l.py.announce, a webpage, a Windows > installer, and so on. What more do we need? Two points: * First of all, we've been a bit sloppy in using "dev release" to mean different things at different points in the discussion. Here, I'm using it the way Skip has been using it recently, to refer to packaged micro releases. * As I've said in other posts, what we're really looking for isn't so much people to download these releases as people to *use* these releases. That means they need to at least feel comfortable submitting bug reports and preferably also feel comfortable creating context diffs and stuff like that. We want to create an environment that eases people like me into this process. We're starting to go in that direction with the Developer's Guide, but I want to keep focused on it. For example, I still can't submit bug reports because Lynx is broken WRT SourceForge, and I haven't yet made the concentrated time it'll take to track down the problem. Granted, this specific issue is somewhat unusual, but I don't think this *type* of problem getting started is unusual. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From mal@lemburg.com Wed Apr 10 16:48:51 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 10 Apr 2002 17:48:51 +0200 Subject: [Python-Dev] Unittests References: <3CB3093C.B7A22727@metaslash.com> <20020409210943.GB16531@panix.com> <3CB3F79D.A143D1C2@lemburg.com> <20020410142824.GC10792@panix.com> Message-ID: <3CB45EE3.E01DD8CE@lemburg.com> Aahz wrote: > > On Wed, Apr 10, 2002, M.-A. Lemburg wrote: > > Aahz wrote: > >> On Tue, Apr 09, 2002, Neal Norwitz wrote: > >>> > >>> whrandom.py > >> > >> This is a deprecated module, no? > > > > I certainly hope not. > > Well, you hope wrong. ;-) Argue with the Timbot. I don't want to argue. Why not have both whrandom and random around ? ... with whrandom using the tools from random ? I don't see the value of deprecating a stone-old module just because there's a new flashy module with a different name around. If it's possible to maintain backward compatibility then it should be maintained. Period. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From guido@python.org Wed Apr 10 16:55:26 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 10 Apr 2002 11:55:26 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Wed, 10 Apr 2002 08:20:40 CDT." References: Message-ID: <200204101555.g3AFtQV22594@pcp742651pcs.reston01.va.comcast.net> > Let's say that I fall into this camp and would like to be able to compile > Python on Windows 98. What good free compiler do you recommend, what are the > minimum steps I need to follow to install it, and what are the minimum steps > I need to follow to compile the latest Python from CVS? (I already know how > to use CVS fairly well.) > > If the only real barrier is a lack of information, let's fix that problem > first. Then we can see whether any Windows users are serious about testing > alphas and betas. This is the only way we'll make any progress. Checking out from CVS is documented well on SF: http://sourceforge.net/cvs/?group_id=5470 If you want a free compiler on Windows, I'd recommend Cygwin (a Unixy development environment that includes GCC). There are some problems with Cygwin, but it's getting better all the time. PythonLabs doesn't have the time to pay attention, so we'd love it if we got help keeping Python running on Cygwin. A few people send us patches for Cygwin regularly, but we have no way to test these, so independent confirmation would be great. I hear there's also a free Borland compiler, but I don't know anything about it. There used to be some Borland specific #ifdefs in the Windows pyconfig.h, but they've probably rotted away. --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@pythoncraft.com Wed Apr 10 16:52:03 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 10 Apr 2002 11:52:03 -0400 Subject: [Python-Dev] Developer resources (was Re: Stability and change) In-Reply-To: References: <20020410041923.GA5501@panix.com> Message-ID: <20020410155203.GA2242@panix.com> On Wed, Apr 10, 2002, Tim Peters wrote: > >>>>> The problem with starship is that there are no criteria to admit new >>>>> users. >>>> >>>> There wasn't any difficulty for me in getting a Starship account a bit >>>> more than a year ago. >>> >>> See? That proves my theory. :-) >> >> And how is that "a problem"? Are you worried about hordes of people >> descending on Starship? > > Guido was making a joke. I laughed. You should too. Ah, that famous Dutch humor. It was clear that the line with the smiley was a joke, not so clear that "the problem" was a joke; I therefore responded seriously. And yes, I did make a point of laughing this time around. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From tim@zope.com Wed Apr 10 17:00:06 2002 From: tim@zope.com (Tim Peters) Date: Wed, 10 Apr 2002 12:00:06 -0400 Subject: [Python-Dev] Unittests In-Reply-To: <20020410142824.GC10792@panix.com> Message-ID: [Aahz] < ,,, > Note to Tim: you haven't followed PEP 4, so you can't really say that > whrandom is deprecated yet. whrandom was deprecated before PEP 4 was so much as a gleam in your loins. If you want to change that, you'll have to get a time machine. From jacobs@penguin.theopalgroup.com Wed Apr 10 17:01:27 2002 From: jacobs@penguin.theopalgroup.com (Kevin Jacobs) Date: Wed, 10 Apr 2002 12:01:27 -0400 (EDT) Subject: [Python-Dev] Re: Cygwin In-Reply-To: <200204101555.g3AFtQV22594@pcp742651pcs.reston01.va.comcast.net> Message-ID: On Wed, 10 Apr 2002, Guido van Rossum wrote: > If you want a free compiler on Windows, I'd recommend Cygwin (a Unixy > development environment that includes GCC). There are some problems > with Cygwin, but it's getting better all the time. The major problem with Cygwin is that it is **slow**. The other problem is that you can't use it for commercial applications. I've gotten about 10 e-mails in reply to my offer to explore getting the Python CVS up and running with MingW. Since Python is already ported to native Win32 APIs, it seems like a viable approach that will not suffer from either of Cygwin's problems (though I have yet to try building a version with threads, or Python more recent than version 1.5.2). I'll keep the list posted on how it goes. -Kevin -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com From skip@pobox.com Wed Apr 10 17:03:34 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 10 Apr 2002 11:03:34 -0500 Subject: [Python-Dev] Multiple revisions [was: Re: Stability and change] In-Reply-To: <200204101540.g3AFeLM22492@pcp742651pcs.reston01.va.comcast.net> References: <3CB44AB7.8010600@strakt.com> <200204101540.g3AFeLM22492@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15540.25174.169519.719473@beluga.mojam.com> Guido> BTW, arch is a lousy name. It's impossible to find using Google. Not to mention which there is a common command by the same name on many Unix systems. I believe Sun was the first company to have an arch command when they began having a bevy of different bits of hardware. It's been around for a long, long time. Skip From paul@prescod.net Wed Apr 10 17:11:22 2002 From: paul@prescod.net (Paul Prescod) Date: Wed, 10 Apr 2002 09:11:22 -0700 Subject: [Python-Dev] Re: Stability and change References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <02040710094202.05787@arthur> <15537.12457.59929.194685@anthem.wooz.org> <15537.43727.369744.356521@grendel.zope.com> <3CB2AE2F.2DAF6373@prescod.net> <200204100109.g3A19s119435@pcp742651pcs.reston01.va.comcast.net> <3CB3B23A.2277F6CE@prescod.net> <200204101333.g3ADXZw22133@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CB4642A.906795A4@prescod.net> Guido van Rossum wrote: > >... > > I didn't say that. NumPy has exactly the same issues. We've lost the thread of discussion and are now in the land of wasted breath, but just in case you are curious how we got here... Fred said: "Keeping PyXML compatible with old versions of Python is a pain." Then he said: "It's enough to make me think that Andrew was right about including XML in Python." To me, that's a non sequiter. PyXML has the same backwards compatibility headache as any other third party module. If I had a module called PyGraphs that needed weak references, I would have a problem if I wanted it to work on Python 2.0 or 1.5.2. Then you said: "Yeah, we only included XML because some people were very vocal about including it for political reasons." Of course I was one of those people so I still wanted to get to the bottom of this non sequiter. PyXML would have exactly the same backwards compatibility headaches if there were no XML support in Python whatsoever. It's just a problem that third party modules have. You either endure the pain or you make your users upgrade to a new version of Python. As far as I can tell, those are the only two choices. Paul Prescod From mal@lemburg.com Wed Apr 10 17:17:59 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 10 Apr 2002 18:17:59 +0200 Subject: [Python-Dev] Regression in unicodestr.encode()? References: <15539.16595.651881.216153@yyz.zope.com> <2mpu182bep.fsf@starship.python.net> Message-ID: <3CB465B7.AF2B80B4@lemburg.com> Michael Hudson wrote: > > barry@zope.com (Barry A. Warsaw) writes: > > > Is the utf-8 encoder in cvs broken? > > I seem to remember that test_unicode failed the last time I built a > wide unicode build from the trunk. Someone else can find the time to > try this again, or I'll do it in a few days... I know that it failed on the maintenance branch, but that is fixed now (it had to do with the way character positions where counted). Does it also fail with CVS ? (that implementation is somewhat different) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From neal@metaslash.com Wed Apr 10 17:16:38 2002 From: neal@metaslash.com (Neal Norwitz) Date: Wed, 10 Apr 2002 12:16:38 -0400 Subject: [Python-Dev] Re: Unittest list References: Message-ID: <3CB46566.C9DC62B7@metaslash.com> Hernan Martinez Foffani wrote: > > From: Neal Norwitz > > I pulled it from where the sun don't shine. :-) > > I did a bunch of ls/grep/echo/... commands. > > Then post processed manually. > > > > It's not maintained anywhere. It would be great if there was > > a place to maintain/update it. Suggestions? > > > > I have another script which builds python with test-coverage > > and will tell you each line/function that doesn't get executed > > in case you get really bored. :-) > > I'm interested in those scripts. Are they available for the > public? Since I don't think most people care, I will mail you directly. If there are enough people that want the script, I'll send it to the mailing list also. > > There's only about 200 functions in Objects/*.c. > > That "200", is the aprox number of functions that don't have > a corresponding unittest? It's the # of C functions that were never called after running the entire regression test suite. > By the way, is there a simmilar package that do the same > thing (tell which line/function that doesn't get executed) > for pure python programs? I mean, something that can run > with the standard Python distrib. There's 2 modules I know of: http://manatee.mojam.com/~skip/python/trace.py http://www.garethrees.org/2001/12/04/python-coverage/coverage.py Neal From skip@pobox.com Wed Apr 10 17:28:00 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 10 Apr 2002 11:28:00 -0500 Subject: [Python-Dev] Unittests In-Reply-To: <3CB45EE3.E01DD8CE@lemburg.com> References: <3CB3093C.B7A22727@metaslash.com> <20020409210943.GB16531@panix.com> <3CB3F79D.A143D1C2@lemburg.com> <20020410142824.GC10792@panix.com> <3CB45EE3.E01DD8CE@lemburg.com> Message-ID: <15540.26640.454643.389710@beluga.mojam.com> mal> Why not have both whrandom and random around ? ... with whrandom mal> using the tools from random ? If I understood Tim correctly yesterday, whrandom's seed method was pretty bad. Migrating the whrandom algorithms to the random module made sense, and having a seed method made sense, but it had to be an improved seed method. If you're going to create a semantically different seed method, you might as well deprecate whrandom altogether. Allowing people to continue using whrandom with the old seed method gives them a false sense of security that they have a good random number generator at their disposal. Using it with a new and improved seed method may allow their code to break in subtle ways. To make matters worse, most people don't even realize they are calling seed in the first place. The best course is simply to deprecate whrandom so people have to explicitly address the issue. Explicit is better than implicit. That said, when I mapped "whrandom" to "random" in my own code, I completely ignored this issue with no problems. Of course, my use of random numbers is not all that demanding. When I multi-threaded my XML-RPC server I had to address locking issues though. Most people will probably be able to execute s/whrandom/random/g and be done with it. Those people who aren't able to make that simple translation are probably rare and know what the issues are. mal> I don't see the value of deprecating a stone-old module mal> just because there's a new flashy module with a different mal> name around. If it's possible to maintain backward mal> compatibility then it should be maintained. Period. It's not just a different name. It's subtly different semantics. Skip P.S. I Trust Tim (tm). From jason@tishler.net Wed Apr 10 17:46:21 2002 From: jason@tishler.net (Jason Tishler) Date: Wed, 10 Apr 2002 12:46:21 -0400 Subject: [Python-Dev] Re: Cygwin In-Reply-To: References: <200204101555.g3AFtQV22594@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020410164621.GA1132@tishler.net> Kevin, On Wed, Apr 10, 2002 at 12:01:27PM -0400, Kevin Jacobs wrote: > The other problem is that you can't use it for commercial applications. Why? The Cygwin license just requires distributing the source (to cygwin1.dll and the linked applications). By "commercial" do you really mean proprietary? If so, then if you don't need shared extensions then you are OK. If you do, then you are not. However, I should note that IANAL. Jason From niemeyer@conectiva.com Wed Apr 10 17:47:30 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Wed, 10 Apr 2002 13:47:30 -0300 Subject: [Python-Dev] Multiple revisions [was: Re: Stability and change] In-Reply-To: <200204101540.g3AFeLM22492@pcp742651pcs.reston01.va.comcast.net> References: <3CB44AB7.8010600@strakt.com> <200204101540.g3AFeLM22492@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020410134730.B2923@ibook.distro.conectiva> > > Perforce, Bitkeeper and Subversion are all "better-breed-CVS" solutions, > > while I consider arch to be a "better-breed-revision-control-system". > > You may be right about Perforce or Subversion (though I doubt it), but > the arch blurb I found sounds very similar to Bitkeeper. It's not just similar. One of the reasons why Bitkeeper doesn't have its code available anymore is because they claimed that Arch was copying the internal mechanisms of Bitkeeper. I have seen both working and would definitely stick for Bitkeeper. OTOH, CVS is good enough, Bitkeeper is not free software, and that's not the place to discuss that . -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From mwh@python.net Wed Apr 10 17:55:11 2002 From: mwh@python.net (Michael Hudson) Date: 10 Apr 2002 17:55:11 +0100 Subject: [Python-Dev] Regression in unicodestr.encode()? In-Reply-To: "M.-A. Lemburg"'s message of "Wed, 10 Apr 2002 18:17:59 +0200" References: <15539.16595.651881.216153@yyz.zope.com> <2mpu182bep.fsf@starship.python.net> <3CB465B7.AF2B80B4@lemburg.com> Message-ID: <2mit6zpk7k.fsf@starship.python.net> "M.-A. Lemburg" writes: > Michael Hudson wrote: > > > > barry@zope.com (Barry A. Warsaw) writes: > > > > > Is the utf-8 encoder in cvs broken? > > > > I seem to remember that test_unicode failed the last time I built a > > wide unicode build from the trunk. Someone else can find the time to > > try this again, or I'll do it in a few days... > > I know that it failed on the maintenance branch, but that is > fixed now (it had to do with the way character positions where > counted). Yes, and forgetting to update pointers... > Does it also fail with CVS ? (that implementation > is somewhat different) That's what I was trying to say. I may have been imagining things. Cheers, M. -- I'll write on my monitor fifty times 'I must not post self-indulgent wibble nobody is interested in to ucam.chat just because I'm bored and I can't find the bug I'm supposed to fix'. -- Steve Kitson, ucam.chat From skip@pobox.com Wed Apr 10 18:00:56 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 10 Apr 2002 12:00:56 -0500 Subject: [Python-Dev] Re: Unittest list In-Reply-To: <3CB46566.C9DC62B7@metaslash.com> References: <3CB46566.C9DC62B7@metaslash.com> Message-ID: <15540.28616.808690.154553@beluga.mojam.com> To get test coverage with gcc and gcov all you need to do is mkdir build.gcov cd build.gcov OPT="-fprofile-arcs -ftest-coverage" ../configure make make quicktest for f in ../{Objects,Python,Modules}/*.c ; do gcov -o . $f done I don't know if that's a script or not. I guess maybe it is... :-) >> By the way, is there a simmilar package that do the same thing for >> pure python programs? Neal> There's 2 modules I know of: Neal> http://manatee.mojam.com/~skip/python/trace.py Neal> http://www.garethrees.org/2001/12/04/python-coverage/coverage.py My trace.py stuff is actually in the distribution these days at Tools/scripts/trace.py. It's languished since I first ripped off profile.py to create it. Skip From jacobs@penguin.theopalgroup.com Wed Apr 10 18:03:16 2002 From: jacobs@penguin.theopalgroup.com (Kevin Jacobs) Date: Wed, 10 Apr 2002 13:03:16 -0400 (EDT) Subject: [Python-Dev] Re: Cygwin In-Reply-To: <20020410164621.GA1132@tishler.net> Message-ID: On Wed, 10 Apr 2002, Jason Tishler wrote: > On Wed, Apr 10, 2002 at 12:01:27PM -0400, Kevin Jacobs wrote: > > The other problem is that you can't use it for commercial applications. > > Why? The Cygwin license just requires distributing the source (to > cygwin1.dll and the linked applications). By "commercial" do you really > mean proprietary? If so, then if you don't need shared extensions then > you are OK. If you do, then you are not. Sorry, I did mean non-open-source and commercial. The closed-source license for Cygwin is several tens of thousands of dollars (I know -- one project I used to work on bought a license and then never used it since I got the MingW version working for them). -Kevin -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com From mal@lemburg.com Wed Apr 10 18:14:46 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 10 Apr 2002 19:14:46 +0200 Subject: [Python-Dev] Regression in unicodestr.encode()? References: <15539.16595.651881.216153@yyz.zope.com> Message-ID: <3CB47306.BD48C78B@lemburg.com> "Barry A. Warsaw" wrote: > > I'm porting over the latest email package to Python 2.3cvs, and I've > had one of my tests fail. I've narrowed it down to the following test > case: > > a = u'\u6b63\u78ba\u306b\u8a00\u3046\u3068\u7ffb\u8a33\u306f\u3055\u308c\u3066\u3044\u307e\u305b\u3093\u3002\u4e00\u90e8\u306f\u30c9\u30a4\u30c4\u8a9e\u3067\u3059\u304c\u3001\u3042\u3068\u306f\u3067\u305f\u3089\u3081\u3067\u3059\u3002\u5b9f\u969b\u306b\u306f\u300cWenn ist das Nunstuck git und' > print repr(a.encode('utf-8', 'replace')) > > In Python 2.2.1 I get > > '\xe6\xad\xa3\xe7\xa2\xba\xe3\x81\xab\xe8\xa8\x80\xe3\x81\x86\xe3\x81\xa8\xe7\xbf\xbb\xe8\xa8\xb3\xe3\x81\xaf\xe3\x81\x95\xe3\x82\x8c\xe3\x81\xa6\xe3\x81\x84\xe3\x81\xbe\xe3\x81\x9b\xe3\x82\x93\xe3\x80\x82\xe4\xb8\x80\xe9\x83\xa8\xe3\x81\xaf\xe3\x83\x89\xe3\x82\xa4\xe3\x83\x84\xe8\xaa\x9e\xe3\x81\xa7\xe3\x81\x99\xe3\x81\x8c\xe3\x80\x81\xe3\x81\x82\xe3\x81\xa8\xe3\x81\xaf\xe3\x81\xa7\xe3\x81\x9f\xe3\x82\x89\xe3\x82\x81\xe3\x81\xa7\xe3\x81\x99\xe3\x80\x82\xe5\xae\x9f\xe9\x9a\x9b\xe3\x81\xab\xe3\x81\xaf\xe3\x80\x8cWenn ist das Nunstuck git und' > > but in Python 2.3 cvs I get > > '\xe6\xad\xa3\xe7\xa2\xba\xe3\x81\xab\xe8\xa8\x80\xe3\x81\x86\xe3\x81\xa8\xe7\xbf\xbb\xe8\xa8\xb3\xe3\x81\xaf\xe3\x81\x95\xe3\x82\x8c\xe3\x81\xa6\xe3\x81\x84\xe3\x81\xbe\xe3\x81\x9b\xe3\x82\x93\xe3\x80\x82\xe4\xb8\x80\xe9\x83\xa8\xe3\x81\xaf\xe3\x83\x89\xe3\x82\xa4\xe3\x83\x84\xe8\xaa\x9e\xe3\x81\xa7\xe3\x81\x99\xe3\x81\x8c\xe3\x80\x81\xe3\x81\x82\xe3\x81\xa8\xe3\x81\xaf\xe3\x81\xa7\xe3\x81\x9f\xe3\x82\x89\xe3\x82\x81\xe3\x81\xa7\xe3\x81\x99\xe3\x80\x82\xe5\xae\x9f\xe9\x9a\x9b\xe3\x81\xab\xe3\x81\xaf\xe3\x80\x8cWenn ist das Nunstuck git u\x00\x00' > > Note that the last two characters, which should be `n' and `d' are now > NULs. My very limited Tim-enlightened understanding is that encoding > a string to UTF-8 should never produce a string with NULs. > > Is the utf-8 encoder in cvs broken? Some debugging with gdb indicates that the codec is indeed writing the 'nd', but the final _PyString_Resize() (which allocates a new buffer and copies the data into that buffer) fails to copy the last two characters from the string or overwrites it with NULLs. Looks like a pymalloc problem to me. Tim ? In any case, I'll checkin a test case for this. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From nas@python.ca Wed Apr 10 18:48:12 2002 From: nas@python.ca (Neil Schemenauer) Date: Wed, 10 Apr 2002 10:48:12 -0700 Subject: [Python-Dev] Stability and Change In-Reply-To: <15539.48208.510028.637885@12-248-41-177.client.attbi.com>; from skip@pobox.com on Tue, Apr 09, 2002 at 11:15:12PM -0500 References: <200204100306.g3A36CE20294@pcp742651pcs.reston01.va.comcast.net> <15539.48208.510028.637885@12-248-41-177.client.attbi.com> Message-ID: <20020410104812.A1445@glacier.arctrix.com> Skip Montanaro wrote: > From where I sit this seems untenable, simply because what you're proposing > is going to require having three or four release managers at the same time, > each managing bugfixes and (what was the term?) "minor features" for a > previous release. Ain't no way I'm going to backport a change to three > previous releases. Good point. If developing Python becomes more tedious we are going to end up with less developers. I know my interest will wane if I have to backport my changes to two or three older branches. Frankly, it seems to me that the people complaining about the rate of change are unwilling to put their money where their mouth. Why should I volunteer my time to help them when I get no benefit from it? Python is stable enough for me. I have no trouble making my code or the approximately 100,000 SLOC written by the MEMS Exchange work with the latest release. Based on resources available the Python project I think we are doing a nearly optimal job. Neil From mal@lemburg.com Wed Apr 10 18:48:27 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 10 Apr 2002 19:48:27 +0200 Subject: [Python-Dev] Unittests References: <3CB3093C.B7A22727@metaslash.com> <20020409210943.GB16531@panix.com> <3CB3F79D.A143D1C2@lemburg.com> <20020410142824.GC10792@panix.com> <3CB45EE3.E01DD8CE@lemburg.com> <15540.26640.454643.389710@beluga.mojam.com> Message-ID: <3CB47AEB.65B12B3E@lemburg.com> Skip Montanaro wrote: > > mal> Why not have both whrandom and random around ? ... with whrandom > mal> using the tools from random ? > > If I understood Tim correctly yesterday, whrandom's seed method was pretty > bad. There's nothing bad in the .seed() method. You just have to pass it some sane values (rather than relying on the defaults). > Migrating the whrandom algorithms to the random module made sense, and > having a seed method made sense, but it had to be an improved seed method. > If you're going to create a semantically different seed method, you might as > well deprecate whrandom altogether. Allowing people to continue using > whrandom with the old seed method gives them a false sense of security that > they have a good random number generator at their disposal. Using it with a > new and improved seed method may allow their code to break in subtle ways. > To make matters worse, most people don't even realize they are calling seed > in the first place. The best course is simply to deprecate whrandom so > people have to explicitly address the issue. Explicit is better than > implicit. > > That said, when I mapped "whrandom" to "random" in my own code, I completely > ignored this issue with no problems. Of course, my use of random numbers is > not all that demanding. When I multi-threaded my XML-RPC server I had to > address locking issues though. Most people will probably be able to execute > > s/whrandom/random/g > > and be done with it. Those people who aren't able to make that simple > translation are probably rare and know what the issues are. You won't have much luck in doing s/whrandom/random/g on a hard-copy Python text book and this is what Python newbies read. I'm not even talking about potential Python users who haven't gotten the slightest idea what sed is... :-) Note that the key point is a different one: every single deprecation causes work (convincing management, code changes, tests, deployment of the new code, training application developers, writing/buying new books). Work costs money. Money causes all sorts of problems. If not absolutely needed in order to make way for new techniques or features, we should leave the old established code in place rather than causing endless support sessions with customers who fear that the code you've written for them 6 months ago starts making not so funny noises or that drag you to update the code on a warranty basis. What's worse, is that management will not be delighted to see that this "new technology we use, called Python" causes quality assurance and training costs every 8-12 months. I wouldn't want Python to go down that road, now that the ball is starting to roll. > mal> I don't see the value of deprecating a stone-old module > mal> just because there's a new flashy module with a different > mal> name around. If it's possible to maintain backward > mal> compatibility then it should be maintained. Period. > > It's not just a different name. It's subtly different semantics. It should be no problem emulating the old behaviour using the new code or simply leaving the old code in place (it's not broken in any way). -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From trentm@ActiveState.com Wed Apr 10 18:59:42 2002 From: trentm@ActiveState.com (Trent Mick) Date: Wed, 10 Apr 2002 10:59:42 -0700 Subject: [Python-Dev] RELEASED: Python 2.2.1 In-Reply-To: ; from mwh@python.org on Wed, Apr 10, 2002 at 11:47:24AM +0100 References: Message-ID: <20020410105940.A13862@ActiveState.com> It might be nice to move the 'r221' CVS tag further along the 'release22-maint' branch. The tree at the current 'r221' location fails to build the docs with the following: make[1]: *** No rule to make target `whatsnew/whatsnew22.tex', needed by `html/whatsnew/whatsnew22.html'. Stop. 'whatsnew22.tex' was added after the tag. Thoughts? Trent [Michael Hudson wrote] > We've released a bugfix release of Python, 2.2.1! > > This release should be fully compatible with Python 2.2. Everyone > using 2.2 is recommended to upgrade to 2.2.1. > > On the 2.2.1 pages > > http://www.python.org/2.2.1/ > > you can find more information, binaries for Windows, Redhat 7.2 & 6.2, > Mandrake 8.2 and MacOS 8, 9 & X, as well as the usual tarball. Thanks > are due to those who provided the binaries and to all who tested the > prerelases during the 2.2.1 release process! > > One nasty bug (that affected Zope in obscure circumstances) and a few > less serious ones have been fixed since the second release candidate. > Two fixes may be of special interest if you use MacOS X: > > - This release finally works on Mac OS X multiprocessors > - The IDE is now fully functional on Mac OS X. > > This being a bugfix release, there are no exciting new features -- we just > fixed a lot of bugs. For a moderately complete list, please see: > > http://www.python.org/2.2.1/NEWS.txt > > Cheers, > The Python development team. > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev -- Trent Mick TrentM@ActiveState.com From skip@pobox.com Wed Apr 10 19:06:46 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 10 Apr 2002 13:06:46 -0500 Subject: [Python-Dev] Unittests In-Reply-To: <3CB47AEB.65B12B3E@lemburg.com> References: <3CB3093C.B7A22727@metaslash.com> <20020409210943.GB16531@panix.com> <3CB3F79D.A143D1C2@lemburg.com> <20020410142824.GC10792@panix.com> <3CB45EE3.E01DD8CE@lemburg.com> <15540.26640.454643.389710@beluga.mojam.com> <3CB47AEB.65B12B3E@lemburg.com> Message-ID: <15540.32566.759039.420345@beluga.mojam.com> It's time for Tim to chime in here. I'm sure he can explain/justify his motives much better than I can. I have never claimed to be able to channel a bot. You stated There's nothing bad in the .seed() method. You just have to pass it some sane values (rather than relying on the defaults). The problem as I see it is that almost everybody who uses whrandom calls the seed method implicitly with the default values. If the defaults are bad choices, then you will get suboptimal sequences. In fact, when Tim wrote the seed method for the WH generator in random.py he didn't just change the defaults. He changed the algorithm used to seed the generator itself. Skip From guido@python.org Wed Apr 10 20:07:38 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 10 Apr 2002 15:07:38 -0400 Subject: [Python-Dev] Re: Unittest list In-Reply-To: Your message of "Wed, 10 Apr 2002 12:16:38 EDT." <3CB46566.C9DC62B7@metaslash.com> References: <3CB46566.C9DC62B7@metaslash.com> Message-ID: <200204101907.g3AJ7ch22996@pcp742651pcs.reston01.va.comcast.net> > > That "200", is the aprox number of functions that don't have > > a corresponding unittest? > > It's the # of C functions that were never called after running > the entire regression test suite. And what percentage is this of the total number of functions there? --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Wed Apr 10 20:13:44 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 10 Apr 2002 15:13:44 -0400 Subject: [Python-Dev] RELEASED: Python 2.2.1 In-Reply-To: <20020410105940.A13862@ActiveState.com> References: <20020410105940.A13862@ActiveState.com> Message-ID: <15540.36584.430237.409927@grendel.zope.com> Trent Mick writes: > It might be nice to move the 'r221' CVS tag further along the > 'release22-maint' branch. The tree at the current 'r221' location fails > to build the docs with the following: > > make[1]: *** No rule to make target `whatsnew/whatsnew22.tex', needed by `html/whatsnew/whatsnew22.html'. Stop. > > 'whatsnew22.tex' was added after the tag. > > Thoughts? This should be fixed! I don't know who did the tagging, but don't remember any coordination activities. This isn't the only file that was changed since whatsnew22.tex was added. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From martin@v.loewis.de Wed Apr 10 20:03:06 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 10 Apr 2002 21:03:06 +0200 Subject: [Python-Dev] Regression in unicodestr.encode()? In-Reply-To: <3CB47306.BD48C78B@lemburg.com> References: <15539.16595.651881.216153@yyz.zope.com> <3CB47306.BD48C78B@lemburg.com> Message-ID: "M.-A. Lemburg" writes: > Some debugging with gdb indicates that the codec is indeed writing > the 'nd', but the final _PyString_Resize() (which allocates a new > buffer and copies the data into that buffer) fails to copy the last > two characters from the string or overwrites it with NULLs. > > Looks like a pymalloc problem to me. Tim ? It's a UTF-8 codec bug. The codec writes over the end of the buffer, then invokes resize. Resizing only copies the allocated bytes, hence the uninitialized bytes at the end. Regards, Martin From trentm@ActiveState.com Wed Apr 10 20:19:42 2002 From: trentm@ActiveState.com (Trent Mick) Date: Wed, 10 Apr 2002 12:19:42 -0700 Subject: [Python-Dev] RELEASED: Python 2.2.1 In-Reply-To: <15540.36584.430237.409927@grendel.zope.com>; from fdrake@acm.org on Wed, Apr 10, 2002 at 03:13:44PM -0400 References: <20020410105940.A13862@ActiveState.com> <15540.36584.430237.409927@grendel.zope.com> Message-ID: <20020410121942.A23370@ActiveState.com> --IS0zKkzwUGydFO0o Content-Type: text/plain; charset=us-ascii Content-Disposition: inline [Fred L . Drake wrote] > This isn't the only file that was changed since whatsnew22.tex was > added. Nope. I have attached the diff (as our Perforce mirror describes it) of changes from 'r221' to the 'release22-maint'-tip. Trent -- Trent Mick TrentM@ActiveState.com --IS0zKkzwUGydFO0o Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="r221_to_release22maint.txt" Change 38383 by trentm@trentm-screwdriver on 2002/04/10 11:08:47 Update the core Python source to the current CVS 'release22-maint' tag tip because CVS at the 'r221' tag fails to build the docs. Affected files ... ... //depot/main/Apps/ActivePython-2_2/src/Core/Doc/ref/ref5.tex#4 integrate ... //depot/main/Apps/ActivePython-2_2/src/Core/Doc/whatsnew/whatsnew22.tex#1 branch ... //depot/main/Apps/ActivePython-2_2/src/Core/Misc/NEWS#4 integrate ... //depot/main/Apps/ActivePython-2_2/src/Core/PCbuild/BUILDno.txt#4 integrate Differences ... ==== //depot/main/Apps/ActivePython-2_2/src/Core/Doc/ref/ref5.tex#4 (text) ==== Index: main/Apps/ActivePython-2_2/src/Core/Doc/ref/ref5.tex --- main/Apps/ActivePython-2_2/src/Core/Doc/ref/ref5.tex.~1~ Wed Apr 10 12:27:42 2002 +++ main/Apps/ActivePython-2_2/src/Core/Doc/ref/ref5.tex Wed Apr 10 12:27:42 2002 @@ -829,7 +829,7 @@ The operators \code{<}, \code{>}, \code{==}, \code{>=}, \code{<=}, and \code{!=} compare the values of two objects. The objects need not have the same type. -If both are numbers, they are coverted to a common type. Otherwise, +If both are numbers, they are converted to a common type. Otherwise, objects of different types \emph{always} compare unequal, and are ordered consistently but arbitrarily. ==== //depot/main/Apps/ActivePython-2_2/src/Core/Misc/NEWS#4 (text) ==== Index: main/Apps/ActivePython-2_2/src/Core/Misc/NEWS --- main/Apps/ActivePython-2_2/src/Core/Misc/NEWS.~1~ Wed Apr 10 12:27:42 2002 +++ main/Apps/ActivePython-2_2/src/Core/Misc/NEWS Wed Apr 10 12:27:42 2002 @@ -1,5 +1,5 @@ What's New in Python 2.2.1 final? -Release date: XX-Apr-2002 +Release date: 10-Apr-2002 ================================= Core and builtins ==== //depot/main/Apps/ActivePython-2_2/src/Core/PCbuild/BUILDno.txt#4 (text) ==== Index: main/Apps/ActivePython-2_2/src/Core/PCbuild/BUILDno.txt --- main/Apps/ActivePython-2_2/src/Core/PCbuild/BUILDno.txt.~1~ Wed Apr 10 12:27:42 2002 +++ main/Apps/ActivePython-2_2/src/Core/PCbuild/BUILDno.txt Wed Apr 10 12:27:42 2002 @@ -34,7 +34,7 @@ Windows Python BUILD numbers ---------------------------- 34 2.2.1 (final) - 9-Apr-2002 + 10-Apr-2002 33 2.2.1c2 26-Mar-2002 32 2.2.1c1 End of Patch. --IS0zKkzwUGydFO0o-- From mal@lemburg.com Wed Apr 10 20:27:04 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 10 Apr 2002 21:27:04 +0200 Subject: [Python-Dev] Regression in unicodestr.encode()? References: <15539.16595.651881.216153@yyz.zope.com> <3CB47306.BD48C78B@lemburg.com> Message-ID: <3CB49208.14B82557@lemburg.com> "Martin v. Loewis" wrote: > > "M.-A. Lemburg" writes: > > > Some debugging with gdb indicates that the codec is indeed writing > > the 'nd', but the final _PyString_Resize() (which allocates a new > > buffer and copies the data into that buffer) fails to copy the last > > two characters from the string or overwrites it with NULLs. > > > > Looks like a pymalloc problem to me. Tim ? > > It's a UTF-8 codec bug. The codec writes over the end of the buffer, > then invokes resize. Resizing only copies the allocated bytes, hence > the uninitialized bytes at the end. Ah, yes, you're right. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From tim.one@comcast.net Wed Apr 10 20:34:06 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 10 Apr 2002 15:34:06 -0400 Subject: [Python-Dev] Regression in unicodestr.encode()? In-Reply-To: <3CB47306.BD48C78B@lemburg.com> Message-ID: > ... > Looks like a pymalloc problem to me. Tim ? I doubt it, but haven't spent much time on it. If you run Barry's test under a debug build, a call to pymalloc's realloc complains immediately upon entry that the passed-in address suffered overwrites (i.e., whoever malloc'ed or realloc'ed this area the preceding time wrote into positions beyond the number of bytes they asked for): Debug memory block at address p=00864280: 180 bytes originally allocated the 4 pad bytes at p-4 are PYMALLOC_FORBIDDENBYTE, as expected the 4 pad bytes at tail=00864334 are not all PYMALLOC_FORBIDDENBYTE (0xfb): at tail+0: 0x69 *** OUCH at tail+1: 0x74 *** OUCH at tail+2: 0x20 *** OUCH at tail+3: 0x75 *** OUCH the block was made by call #1852047475 to debug malloc/realloc data at p: 00 00 00 00 00 00 00 00 ... 6e 73 74 75 63 6b 20 67 Fatal Python error: bad trailing pad byte That appears to be the "it u" near the end of the desired output (from "git und"). So this isn't just off by 1, it's overwriting by a lot. That may be a pymalloc bug, but the odds don't favor it. From mal@lemburg.com Wed Apr 10 20:44:01 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 10 Apr 2002 21:44:01 +0200 Subject: [Python-Dev] Regression in unicodestr.encode()? References: <15539.16595.651881.216153@yyz.zope.com> <3CB47306.BD48C78B@lemburg.com> <3CB49208.14B82557@lemburg.com> Message-ID: <3CB49601.8A16182A@lemburg.com> "M.-A. Lemburg" wrote: > > "Martin v. Loewis" wrote: > > > > "M.-A. Lemburg" writes: > > > > > Some debugging with gdb indicates that the codec is indeed writing > > > the 'nd', but the final _PyString_Resize() (which allocates a new > > > buffer and copies the data into that buffer) fails to copy the last > > > two characters from the string or overwrites it with NULLs. > > > > > > Looks like a pymalloc problem to me. Tim ? > > > > It's a UTF-8 codec bug. The codec writes over the end of the buffer, > > then invokes resize. Resizing only copies the allocated bytes, hence > > the uninitialized bytes at the end. > > Ah, yes, you're right. That is... instrumenting the codec I get these results: >>> (u'\u6b63\u78ba\u306b\u8a00\u3046\u3068\u7ffb\u8a33\u306f' ... u'\u3055\u308c\u3066\u3044\u307e\u305b\u3093\u3002\u4e00' ... u'\u90e8\u306f\u30c9\u30a4\u30c4\u8a9e\u3067\u3059\u304c' ... u'\u3001\u3042\u3068\u306f\u3067\u305f\u3089\u3081\u3067' ... u'\u3059\u3002\u5b9f\u969b\u306b\u306f\u300cWenn ist das' ... u' Nunstuck git und'.encode('utf-8')) cbWritten=0, cbAllocated=144 cbWritten=3, cbAllocated=144 cbWritten=6, cbAllocated=144 cbWritten=9, cbAllocated=144 ... cbWritten=102, cbAllocated=144 cbWritten=105, cbAllocated=144 cbWritten=108, cbAllocated=144 cbWritten=111, cbAllocated=144 cbWritten=114, cbAllocated=144 cbWritten=117, cbAllocated=144 cbWritten=120, cbAllocated=144 cbWritten=123, cbAllocated=144 cbWritten=126, cbAllocated=144 end of string = 'ck git und' '\xe6\xad\xa3\xe7\xa2\xba\xe3....das Nunstuck git u \x8f' (the last two bytes seem to be random data, they change from run to run) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From aahz@pythoncraft.com Wed Apr 10 20:49:20 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 10 Apr 2002 15:49:20 -0400 Subject: [Python-Dev] Unittests In-Reply-To: <3CB47AEB.65B12B3E@lemburg.com> References: <3CB3093C.B7A22727@metaslash.com> <20020409210943.GB16531@panix.com> <3CB3F79D.A143D1C2@lemburg.com> <20020410142824.GC10792@panix.com> <3CB45EE3.E01DD8CE@lemburg.com> <15540.26640.454643.389710@beluga.mojam.com> <3CB47AEB.65B12B3E@lemburg.com> Message-ID: <20020410194920.GA18150@panix.com> On Wed, Apr 10, 2002, M.-A. Lemburg wrote: > > You won't have much luck in doing s/whrandom/random/g on a > hard-copy Python text book and this is what Python newbies > read. I'm not even talking about potential Python users who > haven't gotten the slightest idea what sed is... :-) What Python textbooks use whrandom? > Note that the key point is a different one: every single > deprecation causes work (convincing management, code changes, > tests, deployment of the new code, training application > developers, writing/buying new books). Work costs money. > Money causes all sorts of problems. Would it be okay with you for whrandom to emit a deprecation warning, but not get deleted until Python 3.0? >From my POV, what you're missing in this debate is the cost of keeping the code. If code exists, people ask questions about it. Random issues keep popping up on c.l.py and the simpler the situation, the easier to explain it. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From martin@v.loewis.de Wed Apr 10 20:32:15 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 10 Apr 2002 21:32:15 +0200 Subject: [Python-Dev] Regression in unicodestr.encode()? In-Reply-To: <3CB49208.14B82557@lemburg.com> References: <15539.16595.651881.216153@yyz.zope.com> <3CB47306.BD48C78B@lemburg.com> <3CB49208.14B82557@lemburg.com> Message-ID: "M.-A. Lemburg" writes: > > It's a UTF-8 codec bug. The codec writes over the end of the buffer, > > then invokes resize. Resizing only copies the allocated bytes, hence > > the uninitialized bytes at the end. > > Ah, yes, you're right. Thanks :-) I think the right fix is to avoid any resizing in the UTF-8 codec; that has bitten way too often now. Instead, it should establish the size of the string first, then perform the actual encoding. Regards, Martin From mal@lemburg.com Wed Apr 10 21:02:56 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 10 Apr 2002 22:02:56 +0200 Subject: [Python-Dev] Regression in unicodestr.encode()? References: <15539.16595.651881.216153@yyz.zope.com> <3CB47306.BD48C78B@lemburg.com> <3CB49208.14B82557@lemburg.com> <3CB49601.8A16182A@lemburg.com> Message-ID: <3CB49A70.E2F14483@lemburg.com> (Please ignore my last posting about the instrumented coded... I hit the send button too early) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From pobrien@orbtech.com Wed Apr 10 21:45:15 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Wed, 10 Apr 2002 15:45:15 -0500 Subject: [Python-Dev] Unittests In-Reply-To: <20020410194920.GA18150@panix.com> Message-ID: [Aahz] > > > > You won't have much luck in doing s/whrandom/random/g on a > > hard-copy Python text book and this is what Python newbies > > read. I'm not even talking about potential Python users who > > haven't gotten the slightest idea what sed is... :-) > > What Python textbooks use whrandom? A couple of weeks ago I replaced code of my own that used whrandom. And I can tell you exactly what originally lead me astray. I had found a routine in Zope similar to what I needed and it used whrandom. A check in Python Essential Reference (first edition) confirmed what I needed to know, made no mention that whrandom was deprecated, and made it appear that going directly to whrandom was a shorter path for the functions I needed. Specifically, the functions choice, randint, random and seed are all listed under whrandom, are not repeated under random, and the description under random says that "The module also exports the choice(), randint(), random(), and uniform() functions from the whrandom module." Even the more recently published "Python Standard Library" book makes no mention of whrandom being deprecated. It says "The random module provides a number of different random number generators. The whrandom module is similar, but it also allow you to create multiple generator objects." (Kind of makes whrandom sound superior, doesn't it?) The Python 2.1 Bible, however, does better. "Prior to Version 2.1, the random module used the whrandom module - which provides much of the same functionality - however, the whrandom module is now deprecated." That said, I'm not opposed to getting rid of cruft. And you'd have to be a really raw newbie not to realize that there are going to be some discrepancies between published books and current reality. --- Patrick K. O'Brien Orbtech From martin@v.loewis.de Wed Apr 10 21:04:13 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 10 Apr 2002 22:04:13 +0200 Subject: [Python-Dev] Regression in unicodestr.encode()? In-Reply-To: <3CB49601.8A16182A@lemburg.com> References: <15539.16595.651881.216153@yyz.zope.com> <3CB47306.BD48C78B@lemburg.com> <3CB49208.14B82557@lemburg.com> <3CB49601.8A16182A@lemburg.com> Message-ID: "M.-A. Lemburg" writes: > > Ah, yes, you're right. > > That is... instrumenting the codec I get these results: [...] > cbWritten=126, cbAllocated=144 > end of string = 'ck git und' > '\xe6\xad\xa3\xe7\xa2\xba\xe3....das Nunstuck git u \x8f' > > (the last two bytes seem to be random data, they change > from run to run) Are you saying it does not write past cbAllocated? It certainly does: Just check then length of the encoded string. In the corrected version, the proper length of the encoded string is 158. Regards, Martin From tim.one@comcast.net Wed Apr 10 22:45:09 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 10 Apr 2002 17:45:09 -0400 Subject: [Python-Dev] Unittests In-Reply-To: <3CB45EE3.E01DD8CE@lemburg.com> Message-ID: > I don't want to argue. Why not have both whrandom and random > around ? The prime reasons against are clutter and confusion. whrandom serves no non-redundant purpose, and the existence of both confuses both oldtimers and newcomers (I get the questions about these; I'm tired of answering them). > I don't see the value of deprecating a stone-old module > just because there's a new flashy module with a different > name around. random.py was introduced in March of 1994; the continued existence of the redundant whrandom.py has been confusing people for more than 8 years now. From guido@python.org Wed Apr 10 23:00:44 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 10 Apr 2002 18:00:44 -0400 Subject: [Python-Dev] Deprecating whrandom; and deprecating in general In-Reply-To: Your message of "Wed, 10 Apr 2002 15:49:20 EDT." <20020410194920.GA18150@panix.com> References: <3CB3093C.B7A22727@metaslash.com> <20020409210943.GB16531@panix.com> <3CB3F79D.A143D1C2@lemburg.com> <20020410142824.GC10792@panix.com> <3CB45EE3.E01DD8CE@lemburg.com> <15540.26640.454643.389710@beluga.mojam.com> <3CB47AEB.65B12B3E@lemburg.com> <20020410194920.GA18150@panix.com> Message-ID: <200204102200.g3AM0iQ23326@pcp742651pcs.reston01.va.comcast.net> [Note changed subject] [MAL] > > You won't have much luck in doing s/whrandom/random/g on a > > hard-copy Python text book and this is what Python newbies > > read. I'm not even talking about potential Python users who > > haven't gotten the slightest idea what sed is... :-) [Aahz] > What Python textbooks use whrandom? > > > Note that the key point is a different one: every single > > deprecation causes work (convincing management, code changes, > > tests, deployment of the new code, training application > > developers, writing/buying new books). Work costs money. > > Money causes all sorts of problems. > > Would it be okay with you for whrandom to emit a deprecation warning, > but not get deleted until Python 3.0? > > From my POV, what you're missing in this debate is the cost of > keeping the code. If code exists, people ask questions about it. > Random issues keep popping up on c.l.py and the simpler the > situation, the easier to explain it. I was going to side with Aahz (keeping deprecated code around has costs too), until I decided to go read PEP 4. PEP 4 specifically says that the status of all deprecated modules must be recorded in that PEP. Apparently we forgot all about that when we documented whrandom as deprecated when 2.1 was first released. PEP 4 doesn't say whether deprecated modules should emit a warning when imported, although it says that deprecation proposals "MAY include a patch for the module's source code to indicate deprecation there as well." PEP 4 also doesn't say how long a deprecated module must continue to exist. It says "It is expected that deprecated modules are included in the Python releases that immediately follows the deprecation;" this suggests at least two releases, which matches the deprecation guidelines for language evolution in PEP 5, which suggests at least a year. (I suggest that PEP 4 be augmented to be more clear about these issues.) PEP 4 requires a rationale for deprecation. Three of the four deprecated modules listed there mention as the rationale that the module is not actively being used, and quote no recent mention in deja.com (now groups.google.com) as evidence. So let's apply this to whrandom. and let's say the deprecation note in the docs was invalid because it wasn't recorded in PEP 4. Now we need a rationale. AFAICT the rationale for deprecating whrandom is that its functionality is a duplicate of the random module, we don't want to have to maintain (and answer questions and bug reports about) both modules, and the name is "wrong" (it refers to a specific random algorithm which is known to be rather weak). That's not a particularly strong rationale: there are only three very old SF bug reports about it, and the revision history has been quiet since January 2001. All changes after July 1998 seem to have been cosmetic, from whitespace normalization to a docstring sweep (this can be used as an argument pro or con deprecation). Groups.google.com shows plenty of recent mention of it in c.l.py (a couple per month), so it doesn't appear that nobody is using it. Suppose we decide to deprecate whrandom. The first opportunity to do so will be in 2.3. (I think we shouldn't deprecate modules in micro releases; the PEP is silent about that but I think it would be a bad idea.) Then it will have to continue to exist for at least a year. Because whrandom is still in active use (see above), I think the one year deprecation period is too short in this case and we should use two years. Maybe the warning should only be introduced in the second year? That probably means (assuming releases every 6 months): 2.3 - deprecate in docs 2.5 - issue warning 2.7 - move to Lib/lib-old Now on to Marc's argument against deprecation: > > If not absolutely needed in order to make way for new techniques > > or features, we should leave the old established code in place > > rather than causing endless support sessions with customers > > who fear that the code you've written for them 6 months ago > > starts making not so funny noises or that drag you to update > > the code on a warranty basis. > > > > What's worse, is that management will not be delighted to > > see that this "new technology we use, called Python" causes > > quality assurance and training costs every 8-12 months. I > > wouldn't want Python to go down that road, now that the > > ball is starting to roll. I think there are two different typical use cases for Python code, and they have different preferences for deprecation (and other) warnings. Use case #1 is what we usually think of. Some programmer is maintaining some Python code. When he upgrades the Python version he is using, being the programmer he probably wants to find out about deprecated features in the new Python version, so that he can fix his code so that it won't break outright in the future. IOW, this programmer is glad that there are deprecation warnings (compared to the alternative, where he had no idea that he was using a deprecated feature until it was removed and his program broke outright). Use case #2 is Marc-Andre's and Fredrik's situation: a company writes a program in Python which runs at a customer's site. Assume the program is not large and self-contained enough to warrant it coming with its own version of Python. The customer has lots of Python code in production, maybe some that they wrote themselves, some that they bought from different places, and maybe even some open source downloads. At some point the customer decides to upgrade their Python -- maybe because they like to be on the bleeding edge for their own code base, maybe because they need this for a new open source download, maybe they believe it will fix some Python problem they've experienced intermittently. Now the program they brought from PythonWare or from eGEnix suddenly starts to emit a warning each time it is run. This is annoying: maybe the warning goes to a log file where unexpected things cause alarms to go off, maybe it shows up in an end user's shell window where it causes confusion, maybe there are a lot of warnings and the extra output is simply annoying. These users probably aren't Python savvy, so they may panic and call customer service. Understandably, nobody is happy. Marc-Andre and Fredrik already know their program issues a warning, and they've fixed it in their own version, and they don't want to waste time telling every user how to disable or ignore the warning, or shipping upgrades. How do we satisfy both use cases? We could turn off deprecation warnings by default. But I suspect that most programmers in use case #1 would probably never turn on deprecation warnings, thereby defeating the point of them. I would like to suggest that instead, professional software distributors like eGenix and PythonWare tweak their distributions to turn of warnings by default. This is easy enough: in the main program, insert import warnings warnings.filterwarnings("ignore") at the top (before other imports). If the code needs to run on Python 2.0 or before, insert the whole thing inside a try: / except ImportError: pass. If you want an easy way to enable warnings, you can invent something else (e.g. check an environment variable). Would this be acceptable? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Apr 10 23:03:09 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 10 Apr 2002 18:03:09 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Wed, 10 Apr 2002 11:21:36 EDT." <20020410152135.GB23597@panix.com> References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> <15539.49195.297507.616138@12-248-41-177.client.attbi.com> <15540.14010.181095.298392@slothrop.zope.com> <20020410143241.GD10792@panix.com> <200204101512.g3AFCXd22433@pcp742651pcs.reston01.va.comcast.net> <20020410152135.GB23597@panix.com> Message-ID: <200204102203.g3AM39L23344@pcp742651pcs.reston01.va.comcast.net> > * As I've said in other posts, what we're really looking for isn't so > much people to download these releases as people to *use* these > releases. That means they need to at least feel comfortable > submitting bug reports and preferably also feel comfortable creating > context diffs and stuff like that. We want to create an environment > that eases people like me into this process. We're starting to go > in that direction with the Developer's Guide, but I want to keep > focused on it. Maybe in addition to a developer's guide we need a tester's guide, focusing on how to test new releases and report bugs. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Wed Apr 10 23:05:46 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 10 Apr 2002 18:05:46 -0400 Subject: [Python-Dev] Unittests In-Reply-To: <3CB47AEB.65B12B3E@lemburg.com> Message-ID: [MAL] > There's nothing bad in the .seed() method. You just have to pass it > some sane values (rather than relying on the defaults). I disagree on both counts. The defaults were fine, but the underlying seed method had severe limitations, as the docs for Random.whseed() explain. Being able to initialize to only one of ~2**24 distinct states means Birthday Paradox surprises only needed O(2**12) expected tries to surface, no matter how good the seed selection was. The newer seed() method allows getting at every one of the possible (for this generator) ~2**45 internal states, and makes distinctness guarantees that are easy to understand and to exploit. The older seed method is preserved for bit-for-bit identical results in Random.whseed(), for those who really need it, but no newcomer who knows what they're doing would want to use it. Since most newcomers don't know what they're doing, it's important that the default method save them from the consequences of failing to make an informed analysis of the older seed algorithm. From aahz@pythoncraft.com Wed Apr 10 23:26:31 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 10 Apr 2002 18:26:31 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204102203.g3AM39L23344@pcp742651pcs.reston01.va.comcast.net> References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> <15539.49195.297507.616138@12-248-41-177.client.attbi.com> <15540.14010.181095.298392@slothrop.zope.com> <20020410143241.GD10792@panix.com> <200204101512.g3AFCXd22433@pcp742651pcs.reston01.va.comcast.net> <20020410152135.GB23597@panix.com> <200204102203.g3AM39L23344@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020410222631.GA4145@panix.com> On Wed, Apr 10, 2002, Guido van Rossum wrote: > >> * As I've said in other posts, what we're really looking for isn't so >> much people to download these releases as people to *use* these >> releases. That means they need to at least feel comfortable >> submitting bug reports and preferably also feel comfortable creating >> context diffs and stuff like that. We want to create an environment >> that eases people like me into this process. We're starting to go >> in that direction with the Developer's Guide, but I want to keep >> focused on it. > > Maybe in addition to a developer's guide we need a tester's guide, > focusing on how to test new releases and report bugs. I'll make that part of the Developer's Guide? Or a separate link off the main page? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker From tim.one@comcast.net Wed Apr 10 23:32:42 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 10 Apr 2002 18:32:42 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <20020410044810.GA4932@lilith.my-fqdn.de> Message-ID: [Gerhard H=E4ring] > I doubt that there's a lot of input from Windows users for alphas/b= etas > for major releases. As explained before, every time we do a prerelease, the Windows downl= oads account for a large majority of prerelease downloads (most recently, = 3x as many .exe as .tgz downloads off of SourceForge for 2.2.1c2). I have = no idea what Windows users do with them, though, as they sure don't seem to r= eport many bugs against prereleases. > The reason is that if they needed to test their software, they'd al= so > need new builds of the third-party packages they use on Windows. Bu= t > some of these are hard to build and most users don't know there are > perfectly good free compilers for Windows. Yes, in the land of the blind the one-eyed man is king . > If it is easier for Tim/the Pythonlabs folks, why not just (for > development snapshorts) put the Windows binaries in a zipfile that = can > be unzipped and will just work? I fail to see why an installer is > absolutely necessary for Windows users. Explained before; the Windows build tree structure is unusable for mo= st Windows end-users, and the installer contains all the intelligence ab= out how to create a Windows install tree structure. This includes not only m= oving files, but renaming them too (chiefly to give readable text files .tx= t extensions instead of leaving them bare). From tim.one@comcast.net Wed Apr 10 23:37:55 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 10 Apr 2002 18:37:55 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Message-ID: FWIW, I would like to see Windows alternatives for Python built 100% out of free tools, from compiler to installer, and would further like to see choices at all steps in the chain. I have no time to give to that, though, and it's ironic to even mention it in a "stability" thread . From tim.one@comcast.net Wed Apr 10 23:45:26 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 10 Apr 2002 18:45:26 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15540.19923.569937.419815@12-248-41-177.client.attbi.com> Message-ID: [Skip Montanaro] > ... > That suggests that minority or not, their collective voice carries some > weight. Not in reality. The only thing that carries weight when the pedal hits the metal is who backs up their complaints with their money (resource, talent, time). Zope Corp is doing that bigtime for Python, so no matter what's *said* here, in the end Python will be exactly 6.1% less stable than Zope Corp is willing to pay for . From jeremy@zope.com Wed Apr 10 23:47:13 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Wed, 10 Apr 2002 18:47:13 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15540.18744.580893.827157@12-248-41-177.client.attbi.com> References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> <15539.49195.297507.616138@12-248-41-177.client.attbi.com> <15540.14010.181095.298392@slothrop.zope.com> <15540.18744.580893.827157@12-248-41-177.client.attbi.com> Message-ID: <15540.49393.31493.247859@slothrop.zope.com> >>>>> "SM" == Skip Montanaro writes: SM> I think I need to write a PEP. Everyone needs to write a PEP, just ask Raymond Hettinger. SM> There is perception and there is reality. I believe the reality SM> is that Python is pretty stable. There seems to be a perception SM> outside of the python-dev community that it is not. In large SM> measure this thread has been more about what to do about SM> perceived instability than actual instability. I agree. SM> You need to make measurements that demonstrate to people that SM> your code base is getting more stable, if only for PR purposes. I think this is the crux of your argument, and the part I have overlooked. It doesn't matter if we make a release or I call my grandmother and chat. We just need a regular measurement interval. Then we can say: "We fixed 16 bugs since Jeremy last talked to Grannie." Or since we made a release, or whatever. SM> Associating those metrics with specific release points would, I SM> think, provide more confidence in those numbers. It's cheap to SM> add a new tag to CVS. [...] SM> Even if not a single soul downloads and builds a particular SM> snapshot, creating such snapshots helps because they provide you SM> with static points in the CVS repository from which to make SM> measurements. We could just use a snapshot generated by date In fact, there's a cron job on SF that generates a nightly tarball. I'll bet no one has ever downloaded one. SM> I'm not suggesting that PythonLabs take on the extra burden of SM> making a bug of measurements. Just provide the tickmarks for SM> the graph. I don't see any value to producing more tarballs or tags or releases in order to mark the occasion of measuring progress on fixing bugs. Just make the measurement. BTW, what measurements are you proposing to make? How will you make them? What will they tell us? Jeremy From niemeyer@conectiva.com Thu Apr 11 00:15:47 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Wed, 10 Apr 2002 20:15:47 -0300 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15540.49393.31493.247859@slothrop.zope.com> References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> <15539.49195.297507.616138@12-248-41-177.client.attbi.com> <15540.14010.181095.298392@slothrop.zope.com> <15540.18744.580893.827157@12-248-41-177.client.attbi.com> <15540.49393.31493.247859@slothrop.zope.com> Message-ID: <20020410201547.C23929@ibook.distro.conectiva> Hi Jeremy! [...] > I don't see any value to producing more tarballs or tags or releases > in order to mark the occasion of measuring progress on fixing bugs. > Just make the measurement. > > BTW, what measurements are you proposing to make? How will you make > them? What will they tell us? You may want to have a look at the following URL: http://snapshot.conectiva.com.br This was made for the exact purpose of what is being discussed. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From andy@reportlab.com Thu Apr 11 00:23:30 2002 From: andy@reportlab.com (Andy Robinson) Date: Thu, 11 Apr 2002 00:23:30 +0100 Subject: [Python-Dev] Stability and Change In-Reply-To: <20020410174802.26981.12396.Mailman@mail.python.org> Message-ID: > > Message: 14 > Date: Wed, 10 Apr 2002 10:48:12 -0700 > From: Neil Schemenauer > To: Guido van Rossum , python-dev@python.org > Subject: Re: [Python-Dev] Stability and Change > > Skip Montanaro wrote: > > From where I sit this seems untenable, simply because what > you're proposing > > is going to require having three or four release managers at > the same time, > > each managing bugfixes and (what was the term?) "minor features" for a > > previous release. Ain't no way I'm going to backport a change to three > > previous releases. But that's not the point. Who asked you to backport new features? If the consensus is that 2.1.3 is pretty stable and it is labelled as such, work should be minimal. But it won't happen often. And by definition we don't want the minor features backported; unnecessary changes to the code base would destabilize it. It's absolutely reasonable to find those with an interest in maintaining a stable track and get them to do some of the work. I'm more attuned to helping with packaging and distribution simply because I can't program in C. I think the solution is somewhat outside the core Python team's hands. Those of us who really care about stability need to get together and forge the consensus that we can skip some releases; those of you wanting to forge ahead just need to accept that there are sound reasons for many people to have an upgrade cycle longer than 6 months. Having said that, we just designate the 18-month ones arbitrarily, so that (for example) ReportLab and Zope and eGenix are fairly likely to want the same Python on a customer box in a given quarter. In summary I'd like a note from Guido on the download page saying "we do releases every 6 months but if you prefer a longer cycle, go for 2.1.3 for now, and I will designate a stable 2.4.X release some time in 2003". It doesn't mean 2.2 or 2.3 is bad in any way, but it gives us a decent planning horizon. Maybe EuroPython is the time to pick this up. - Andy Robinson From jeremy@zope.com Thu Apr 11 00:56:53 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Wed, 10 Apr 2002 19:56:53 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <20020410201547.C23929@ibook.distro.conectiva> References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> <15539.49195.297507.616138@12-248-41-177.client.attbi.com> <15540.14010.181095.298392@slothrop.zope.com> <15540.18744.580893.827157@12-248-41-177.client.attbi.com> <15540.49393.31493.247859@slothrop.zope.com> <20020410201547.C23929@ibook.distro.conectiva> Message-ID: <15540.53573.806025.545086@slothrop.zope.com> >>>>> "GN" == Gustavo Niemeyer writes: GN> Hi Jeremy! [...] >> BTW, what measurements are you proposing to make? How will you >> make them? What will they tell us? GN> You may want to have a look at the following URL: GN> http://snapshot.conectiva.com.br GN> This was made for the exact purpose of what is being discussed. I see lots of colors, tiny graphs, and bunches of numbers. Example: There are 15 normal bugs with a weight of 38.668 and an impact of 3.7. Is that supposed to mean something? >From this page, I can't tell how bugs are measured, how they are classified, and what all the numbers mean. Was all this done by hand? It looks a little over-engineered to me. I was imagining the answer would be something more like: Count number of bugs marked as "closed" and "fixed" over some time interval. Jeremy From jeremy@zope.com Thu Apr 11 01:16:35 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Wed, 10 Apr 2002 20:16:35 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15540.53573.806025.545086@slothrop.zope.com> References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> <15539.49195.297507.616138@12-248-41-177.client.attbi.com> <15540.14010.181095.298392@slothrop.zope.com> <15540.18744.580893.827157@12-248-41-177.client.attbi.com> <15540.49393.31493.247859@slothrop.zope.com> <20020410201547.C23929@ibook.distro.conectiva> <15540.53573.806025.545086@slothrop.zope.com> Message-ID: <15540.54755.315835.618339@slothrop.zope.com> >>>>> "JH" == Jeremy Hylton writes: JH> I was imagining the answer would be something more like: Count JH> number of bugs marked as "closed" and "fixed" over some time JH> interval. I think that if the primary purpose of measuring stability is communicating it to a large user population, including managers and decision makers. We need to have only one or two easy to understand metrics. Jeremy From tim.one@comcast.net Thu Apr 11 01:20:17 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 10 Apr 2002 20:20:17 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15540.53573.806025.545086@slothrop.zope.com> Message-ID: [Jeremy Hylton] > ... > I was imagining the answer would be something more like: Count number > of bugs marked as "closed" and "fixed" over some time interval. People are highly suspicious of statistics with a clear meaning. I think it has something to do with penis length measurements (on which count *I* have no cause for fear, let me assure you -- indeed I break most measuring tapes whenever a squirrel runs by outside my window). Is there a simple way to get a count of bugs fixed on the release22-maint branch over a given time span? I don't think so, but there *could* be if people added an entry to the appropriate NEWS file(s) per bug fixed. Or maybe that's just dreamming up a way to discourage bug fixes . From niemeyer@conectiva.com Thu Apr 11 01:26:10 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Wed, 10 Apr 2002 21:26:10 -0300 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <15540.53573.806025.545086@slothrop.zope.com> References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> <15539.49195.297507.616138@12-248-41-177.client.attbi.com> <15540.14010.181095.298392@slothrop.zope.com> <15540.18744.580893.827157@12-248-41-177.client.attbi.com> <15540.49393.31493.247859@slothrop.zope.com> <20020410201547.C23929@ibook.distro.conectiva> <15540.53573.806025.545086@slothrop.zope.com> Message-ID: <20020410212610.A27357@ibook.distro.conectiva> > GN> This was made for the exact purpose of what is being discussed. > > I see lots of colors, tiny graphs, and bunches of numbers. Example: > There are 15 normal bugs with a weight of 38.668 and an impact of > 3.7. Is that supposed to mean something? :-)) > >From this page, I can't tell how bugs are measured, how they are > classified, and what all the numbers mean. Well, but it may give you an idea. Older bugs have more impact than newer ones, blocker bugs have more impact than enhancements, and so on. > Was all this done by hand? No, it's connected with our bugzilla. > It looks a little over-engineered to me. Maybe. But it came to solve the same problem you were talking about, and it worked (indeed, we even got some free advertisement at the time this was first published). We needed to show our users the distribution is actively maintained, and that we care about submitted bugs (now it doesn't show that because we've just released a new version, as the graph shows). > I was imagining the answer would be something more like: Count number > of bugs marked as "closed" and "fixed" over some time interval. That's why I suggested that page. IMO, it's not just about counting the number of bugs. You have to know what they mean: how long they are around, if they are about something important, maybe it's just an enhancement purpose, etc. That's what the "weight" and "impact" means. Just counting bugs may, at least in our case, give the wrong idea about what really happens. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From akuchlin@mems-exchange.org Thu Apr 11 01:37:43 2002 From: akuchlin@mems-exchange.org (akuchlin@mems-exchange.org) Date: Wed, 10 Apr 2002 20:37:43 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: ; from tim.one@comcast.net on Wed, Apr 10, 2002 at 08:20:17PM -0400 References: <15540.53573.806025.545086@slothrop.zope.com> Message-ID: <20020410203743.A25225@mems-exchange.org> On Wed, Apr 10, 2002 at 08:20:17PM -0400, Tim Peters wrote: >Is there a simple way to get a count of bugs fixed on the release22-maint >branch over a given time span? I don't think so, but there *could* be if >people added an entry to the appropriate NEWS file(s) per bug fixed. Or >maybe that's just dreamming up a way to discourage bug fixes . Oh, sure; it's straightforward to write a script to pull log messages from a given branch, and if people religiously put 'bug #NNN' in their log messages, count up the fixed bugs. Handling bugs fixed per time would require more script hacking, but nothing terrifying. I've done this ever since whatsnew21. My numbers, surely underestimates, are: 2.1: 117 patches, 136 bugs 2.2: 527 patches, 683 bugs 2.2.1: 139 patches, 143 bugs --amk (www.amk.ca) If he was a mortician, the corpses would keep their eyes open. -- Glitz, in "Dragonfire" From guido@python.org Thu Apr 11 01:40:59 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 10 Apr 2002 20:40:59 -0400 Subject: [Python-Dev] Stability and Change In-Reply-To: Your message of "Thu, 11 Apr 2002 00:23:30 BST." References: Message-ID: <200204110040.g3B0exf24293@pcp742651pcs.reston01.va.comcast.net> > It's absolutely reasonable to find those with an interest > in maintaining a stable track and get them to do some of the > work. I'm more attuned to helping with packaging and > distribution simply because I can't program in C. That's fine. We need all the help we can get! > I think the solution is somewhat outside the core Python team's > hands. Yup, those hands are already full. > Those of us who really care about stability need to get together and > forge the consensus that we can skip some releases; those of you > wanting to forge ahead just need to accept that there are sound > reasons for many people to have an upgrade cycle longer than 6 > months. I accept it. > Having said that, we just designate the 18-month ones arbitrarily, > so that (for example) ReportLab and Zope and eGenix are fairly > likely to want the same Python on a customer box in a given quarter. Do you think it's necessary to synchronize between different companies? Feel free to try, but there could be all sorts of reasons why you may differ. Since you can have several different minor (but not micro) releases on the same system (at least Unix) it shouldn't be a big deal -- if you require Python 2.x, just put #! /usr/bin/env python2.x at the top of your files. > In summary I'd like a note from Guido on the download page > saying "we do releases every 6 months but if you prefer a longer > cycle, go for 2.1.3 for now, and I will designate a stable 2.4.X > release some time in 2003". It doesn't mean 2.2 or 2.3 is bad > in any way, but it gives us a decent planning horizon. Do I really have to spell it out? Also, I don't want to *recommend* any particular release. I'd like to just say "pick any release you're comfortable with, and stick with it as long as you like." > Maybe EuroPython is the time to pick this up. I'll be there! --Guido van Rossum (home page: http://www.python.org/~guido/) From niemeyer@conectiva.com Thu Apr 11 01:38:19 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Wed, 10 Apr 2002 21:38:19 -0300 Subject: [Python-Dev] Re: Stability and change In-Reply-To: References: <15540.53573.806025.545086@slothrop.zope.com> Message-ID: <20020410213818.A29081@ibook.distro.conectiva> > People are highly suspicious of statistics with a clear meaning. I think it > has something to do with penis length measurements (on which count *I* have > no cause for fear, let me assure you -- indeed I break most measuring tapes > whenever a squirrel runs by outside my window). No details, please.. :-) > Is there a simple way to get a count of bugs fixed on the release22-maint > branch over a given time span? I don't think so, but there *could* be if > people added an entry to the appropriate NEWS file(s) per bug fixed. Or > maybe that's just dreamming up a way to discourage bug fixes . Maybe you're just looking at the wrong side. You have a complete bug tracking system, with information about every open and closed bugs. Is it time to ask SF to provide some automated access? -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From guido@python.org Thu Apr 11 01:45:01 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 10 Apr 2002 20:45:01 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Wed, 10 Apr 2002 20:16:35 EDT." <15540.54755.315835.618339@slothrop.zope.com> References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> <15539.49195.297507.616138@12-248-41-177.client.attbi.com> <15540.14010.181095.298392@slothrop.zope.com> <15540.18744.580893.827157@12-248-41-177.client.attbi.com> <15540.49393.31493.247859@slothrop.zope.com> <20020410201547.C23929@ibook.distro.conectiva> <15540.53573.806025.545086@slothrop.zope.com> <15540.54755.315835.618339@slothrop.zope.com> Message-ID: <200204110045.g3B0j1I24344@pcp742651pcs.reston01.va.comcast.net> > I think that if the primary purpose of measuring stability is > communicating it to a large user population, including managers and > decision makers. We need to have only one or two easy to understand > metrics. I don't think we need metrics. I imagine bugs fixed per month is as uninteresting as lines of code to the average PHB (pointy-haired boss), if it's not their own product. They just want us to say "this one's good". I'd be happy to give a guideline that says "all 2.x.0 releases are solid but may contain untested code and surprises; all 2.x.1 and beyond are stabilized." --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Apr 11 01:46:21 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 10 Apr 2002 20:46:21 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Wed, 10 Apr 2002 20:37:43 EDT." <20020410203743.A25225@mems-exchange.org> References: <15540.53573.806025.545086@slothrop.zope.com> <20020410203743.A25225@mems-exchange.org> Message-ID: <200204110046.g3B0kLg24362@pcp742651pcs.reston01.va.comcast.net> > Oh, sure; it's straightforward to write a script to pull log messages > from a given branch, and if people religiously put 'bug #NNN' in their > log messages, count up the fixed bugs. Then someone has to police this until we've all got the religion down. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Apr 11 01:50:52 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 10 Apr 2002 20:50:52 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Your message of "Wed, 10 Apr 2002 21:38:19 -0300." <20020410213818.A29081@ibook.distro.conectiva> References: <15540.53573.806025.545086@slothrop.zope.com> <20020410213818.A29081@ibook.distro.conectiva> Message-ID: <200204110050.g3B0oqB24389@pcp742651pcs.reston01.va.comcast.net> > Maybe you're just looking at the wrong side. You have a complete bug > tracking system, with information about every open and closed bugs. Is > it time to ask SF to provide some automated access? SF does provide reports. E.g. go to http://sourceforge.net/tracker/reporting/?atid=105470&what=aging&span=52&period=week&group_id=5470 and see that recently we've gotten much better at closing bugs quickly, and that the last 6 weeks or so we've started to reduce the number of open bugs (after letting them slip since Dec 2001 :-). --Guido van Rossum (home page: http://www.python.org/~guido/) From nhodgson@bigpond.net.au Thu Apr 11 01:47:36 2002 From: nhodgson@bigpond.net.au (Neil Hodgson) Date: Thu, 11 Apr 2002 10:47:36 +1000 Subject: [Python-Dev] Re: Stability and change References: <200204101555.g3AFtQV22594@pcp742651pcs.reston01.va.comcast.net> Message-ID: <00d301c1e0f2$79b09e50$0acc8490@neil> Guido van Rossum: > I hear there's also a free Borland compiler, but I don't know anything > about it. There used to be some Borland specific #ifdefs in the > Windows pyconfig.h, but they've probably rotted away. I support compiling Scintilla and SciTE with the Borland, Microsoft and GNU (mingw gcc) compilers. There are good reasons for using Microsoft (best optimization, tools, most used compiler for Python on Windows) and GNU (free and will stay free). Borland doesn't optimize as well as the others and may not always be available for free. The only win for Borland is that it has better diagnostics than the others. Neil From tim.one@comcast.net Thu Apr 11 01:47:27 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 10 Apr 2002 20:47:27 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <20020410213818.A29081@ibook.distro.conectiva> Message-ID: [Gustavo Niemeyer] > Maybe you're just looking at the wrong side. You have a complete bug > tracking system, with information about every open and closed bugs. Is > it time to ask SF to provide some automated access? When a bug is closed, nothing in the bug tracker records the release(s) in which it was fixed. There are a variety of fields like "Group" and "Category" with one-choice possibilities, and people are often careless in setting even those. From tim.one@comcast.net Thu Apr 11 02:11:00 2002 From: tim.one@comcast.net (Tim Peters) Date: Wed, 10 Apr 2002 21:11:00 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204110050.g3B0oqB24389@pcp742651pcs.reston01.va.comcast.net> Message-ID: > SF does provide reports. E.g. go to > ... > and see that recently we've gotten much better at closing bugs > quickly, and that the last 6 weeks or so we've started to reduce the > number of open bugs (after letting them slip since Dec 2001 :-). Big caution: the easiest way to get a dramatic improvement in the "Average Turnaround Time For Closed Items" graph is to leave old bugs open forever. That statistic appears to mean exactly what it says! This is why, e.g., if you look at that report by days, you can see it plummet from 27 on 2002-02-2o down to 1.75 on the next day. I thought Mark Hammond was going to kill us when he closed a slew of moldy old Windows bugs last week . From neal@metaslash.com Thu Apr 11 03:09:01 2002 From: neal@metaslash.com (Neal Norwitz) Date: Wed, 10 Apr 2002 22:09:01 -0400 Subject: [Python-Dev] Re: Unittest list References: <3CB46566.C9DC62B7@metaslash.com> <200204101907.g3AJ7ch22996@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CB4F03D.C584533C@metaslash.com> Guido van Rossum wrote: > > > > That "200", is the aprox number of functions that don't have > > > a corresponding unittest? > > > > It's the # of C functions that were never called after running > > the entire regression test suite. > > And what percentage is this of the total number of functions there? In Objects/*.c, I estimate there are 1153 functions by: egrep -A 3 '^(static|_Py|Py)' *.c | egrep '\.c-\{' | wc There are about 161 in my list of untested functions. I created this report around Dec 14, but I don't think this # has changed too much. Although there may be about 10 more functions which were tested, since I remember Fred adding some tests. I don't know if the new tests were reflected. So that's about 14% of functions in Objects/*.c are not called throughout all the regression tests. I will try to run some updated reports to get more accurate info. Some of the modules which had a larger # of untested functions include: Objects/abstract.c Objects/descrobject.c Objects/typeobject.c Objects/unicodeobject.c Objects/weakrefobject.c Neal From fdrake@acm.org Thu Apr 11 03:51:50 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 10 Apr 2002 22:51:50 -0400 Subject: [Python-Dev] Re: Unittest list In-Reply-To: <3CB4F03D.C584533C@metaslash.com> References: <3CB46566.C9DC62B7@metaslash.com> <200204101907.g3AJ7ch22996@pcp742651pcs.reston01.va.comcast.net> <3CB4F03D.C584533C@metaslash.com> Message-ID: <15540.64070.769894.589552@grendel.zope.com> Neal Norwitz writes: > Some of the modules which had a larger # of untested functions include: ... > Objects/weakrefobject.c I'd guess that a lot of untested functions here are the slot wrappers, which are macro generated and all boilerplate, except for a couple. I'd be interested in seeing the list to see if my intuition is right though; if wrong, it's time to add some tests. And it probably wouldn't hurt to add tests for the boilerplate stuff anyway, just to be extra careful about understanding expectations and protecting against regression failures. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From geoff@gerrietts.net Thu Apr 11 03:43:10 2002 From: geoff@gerrietts.net (Geoff Gerrietts) Date: Wed, 10 Apr 2002 19:43:10 -0700 Subject: [Python-Dev] Stability and Change In-Reply-To: <200204110040.g3B0exf24293@pcp742651pcs.reston01.va.comcast.net> References: <200204110040.g3B0exf24293@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020411024310.GA30655@isis.gerrietts.net> Quoting Guido van Rossum (guido@python.org): > Do you think it's necessary to synchronize between different > companies? Feel free to try, but there could be all sorts of reasons > why you may differ. Since you can have several different minor (but > not micro) releases on the same system (at least Unix) it shouldn't be > a big deal -- if you require Python 2.x, just put > > #! /usr/bin/env python2.x > > at the top of your files. I think the advantage to maintaining a single "curmudgeon" release over the space of 18 months or so, is that it gives extension developers a single testing and backporting target. That allows people who have, for one reason or another, a need to stick to one release for some window of time > 6 months, a stable platform that has a decent chance of being well-supported across the board. The presentation for this might be "this release is no more stable than the current release, but is well-supported and will not see major changes". > > In summary I'd like a note from Guido on the download page > > saying "we do releases every 6 months but if you prefer a longer > > cycle, go for 2.1.3 for now, and I will designate a stable 2.4.X > > release some time in 2003". It doesn't mean 2.2 or 2.3 is bad > > in any way, but it gives us a decent planning horizon. > > Do I really have to spell it out? > > Also, I don't want to *recommend* any particular release. I'd like to > just say "pick any release you're comfortable with, and stick with it > as long as you like." In some ways I think that Guido is the /wrong/ person to say this. So would Tim or pretty much anyone else who is identified as a "core developer". The Curmudgeon's Release may enjoy a certain amount of support from the core community, but it should be an external artifact. The rationale here is that the python community should continue to adopt new features, and the development of the language should not halt, and no particular release should be "blessed" as the One True Release Until The Next True Release. The Curmudgeon's Release /should/ be an alternative -- but it would be good if the alternative were somewhat standard, because that would increase the likelihood of widespread support. All that needs to happen is some community who can do it, needs to emerge to do it. My C skills are acceptable, but my experience in compilers and interpreters is minimal and my time sapped on many fronts. I joined the list because I thought I could help with this effort, and I will, but I hesitate to accept responsibility more than a bite-sized piece at a time. > > Maybe EuroPython is the time to pick this up. > > I'll be there! I won't, so please post the results of any discussion :) -- Geoff Gerrietts "Ordinarily he was insane, but he had lucid moments when he was merely stupid." --Heinrich Heine From neal@metaslash.com Thu Apr 11 04:17:24 2002 From: neal@metaslash.com (Neal Norwitz) Date: Wed, 10 Apr 2002 23:17:24 -0400 Subject: [Python-Dev] Re: Unittest list References: <3CB46566.C9DC62B7@metaslash.com> <200204101907.g3AJ7ch22996@pcp742651pcs.reston01.va.comcast.net> <3CB4F03D.C584533C@metaslash.com> <15540.64070.769894.589552@grendel.zope.com> Message-ID: <3CB50044.5EA86440@metaslash.com> "Fred L. Drake, Jr." wrote: > > Neal Norwitz writes: > > Some of the modules which had a larger # of untested functions include: > ... > > Objects/weakrefobject.c > > I'd guess that a lot of untested functions here are the slot wrappers, > which are macro generated and all boilerplate, except for a couple. > I'd be interested in seeing the list to see if my intuition is right > though; if wrong, it's time to add some tests. And it probably > wouldn't hurt to add tests for the boilerplate stuff anyway, just to > be extra careful about understanding expectations and protecting > against regression failures. weakref is a lot better than it used to be. For anyone interested, I'll put the info on the MetaSlash site: http://www.metaslash.com/py/ Specifically for the untested functions in Objects/ http://www.metaslash.com/py/Objects/lines-untested-funcs The script which generates the files can be found here: http://www.metaslash.com/py/make-stats.sh I'll try to keep useful information there. Neal From stephen@xemacs.org Thu Apr 11 04:50:19 2002 From: stephen@xemacs.org (Stephen J. Turnbull) Date: 11 Apr 2002 12:50:19 +0900 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <20020410222631.GA4145@panix.com> References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> <15539.49195.297507.616138@12-248-41-177.client.attbi.com> <15540.14010.181095.298392@slothrop.zope.com> <20020410143241.GD10792@panix.com> <200204101512.g3AFCXd22433@pcp742651pcs.reston01.va.comcast.net> <20020410152135.GB23597@panix.com> <200204102203.g3AM39L23344@pcp742651pcs.reston01.va.comcast.net> <20020410222631.GA4145@panix.com> Message-ID: <87ofgqao78.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "aahz" == aahz writes: aahz> I'll make that part of the Developer's Guide? Or a aahz> separate link off the main page? I favor the separate link. (1) It attracts the attention of the excessively modest who would never consider themselves "developers." (2) It gives a status boost to people who often will go many releases without making a bug report, and tend to feel that what they're doing is "no big deal." But aggregated over scores of downloads, it is. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Don't ask how you can "do" free software business; ask what your business can "do for" free software. From python@rcn.com Thu Apr 11 05:46:43 2002 From: python@rcn.com (Raymond Hettinger) Date: Thu, 11 Apr 2002 00:46:43 -0400 Subject: [Python-Dev] Stability Metrics [was Stability and Change] References: <20020411025313.29814.31653.Mailman@mail.python.org> Message-ID: <003201c1e113$e1aca1e0$0cb53bd0@othello> From: akuchlin@mems-exchange.org > Oh, sure; it's straightforward to write a script to pull log messages > from a given branch, and if people religiously put 'bug #NNN' in their > log messages, count up the fixed bugs. Handling bugs fixed per time > would require more script hacking, but nothing terrifying. > > I've done this ever since whatsnew21. My numbers, surely > underestimates, are: > > 2.1: 117 patches, 136 bugs > 2.2: 527 patches, 683 bugs > 2.2.1: 139 patches, 143 bugs Whoa there! The conversation is now trending toward bugs as a metric of merit rather than demerit. Of course, fixing bugs is good, but not having them in the first place is better. Before choosing a publicity metric, consider the significance of the metric if it is very large or very small. In the above example, 0 bugs would likely indicate that no maintenance is taking place. Having 10,000 bugs would indicate that the release process was a disaster. Whatever metric is choosen (and I DO think having a stability metric is good), it should have a clear interpretation that a higher number is good and a lower number is bad or vice-versa. it-takes-an-accountant-to-notice-these-things-ly yours, Raymond Hettinger, CPA From skip@pobox.com Thu Apr 11 06:02:40 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 11 Apr 2002 00:02:40 -0500 Subject: [Python-Dev] Can the cgi module be made Unicode-aware? Message-ID: <15541.6384.351664.886436@12-248-41-177.client.attbi.com> I keep trying to handle various places in my code where I can get input in non-ASCII encodings. Today I realized the cgi module does nothing to translate Unicode data into unicode objects. I see in one instance that I am getting data that is clearly utf-8 encoded, but I see nothing in the CGI script's environment variables to suggest the client web browser told the server how the data was encoded other than the obvious "Content-Type: application/x-www-form-urlencoded". Is utf-8 implied for the data once the url encoding has been reversed? Should the cgi module be made Unicode-aware? If so, how? I can never remember the incantation to convert non-ASCII string objects to Unicode objects and nothing I've tried by trial-and-error so far works. I *don't* want to adopt the workaround outlined in FAQ question 4.102 (change the default site-wide encoding). Perhaps that question should be extended with more appropriate information about converting raw strings with non-ASCII content to unicode. Skip From greg@cosc.canterbury.ac.nz Thu Apr 11 06:24:37 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 11 Apr 2002 17:24:37 +1200 Subject: [Python-Dev] ANN: Pyrex 0.1.1 Message-ID: <3CB51E15.81F04B2D@cosc.canterbury.ac.nz> I have fixed a few bugs in Pyrex and made it available as version 0.1.1: http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/ While you're there, pick up the latest version of Plex too (1.1.2), which doesn't use yield as a method name any more. Fixes in this version: * Bogus re-use of temp variables shouldn't happen now. * Access to builtin names now works. * Exceptions should work now (in simple cases, anyway). * Conditions in if-statements now evaluated correctly. More bug fixes are in the works. Thanks to all who have provided feedback so far -- keep it coming! -- Greg Ewing, Computer Science Dept, University of Canterbury, Christchurch, New Zealand To get my email address, please visit my web page: http://www.cosc.canterbury.ac.nz/~greg From skip@pobox.com Thu Apr 11 06:46:49 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 11 Apr 2002 00:46:49 -0500 Subject: [Python-Dev] Stability and Change In-Reply-To: References: <20020410174802.26981.12396.Mailman@mail.python.org> Message-ID: <15541.9033.825422.207087@12-248-41-177.client.attbi.com> >> > From where I sit this seems untenable, simply because what you're >> > proposing is going to require having three or four release managers >> > at the same time, each managing bugfixes and (what was the term?) >> > "minor features" for a previous release. Ain't no way I'm going to >> > backport a change to three previous releases. The "you" I was responding to was Guido, and his notion that several back versions of Python code be maintained for bug fixes. That is what I think is untenable in the current environment. Andy> But that's not the point. Who asked you to backport new features? Nobody. You told Guido you wanted more stability and he responded with: Guido> How about this as a compromise: each major release (which for Guido> Python is 2.0, 2.1, 2.2, 2.3 etc., never mind that only the Guido> "minor" digit varies -- that's how we number 'em) has a Guido> shelf-life of at least three subsequent major releases or 24 Guido> months, whichever is shorter, but never less than 18 months. Guido> Shelf-life means that we follow up with bugfix releases (like Guido> 2.1.1, 2.1.2, 2.1.3) to maintain and improve stability, if Guido> necessary back-porting selected minor features, ... Note the proposed committment to each major release having a preset shelf life. Also note the phrase at the end about possibly back-porting selected minor features. That's what I was responding to. Now, I suspect you would be happier with Guido's proposal than the status quo. Maybe a bunch of extra people will emerge from the woodwork to implement it, but I'm skeptical. Guaranteeing the shelf-life of releases for potentially two years, when PythonLabs typically releases every six months, means you're going to have four active releases going at any one time. That means every bug fix that is applied to the main CVS trunk has to at least be considered for backporting to all those other releases, maybe not applied, but at least considered. Practically speaking, that means you need a release manager for each of those active releases, and they have (or a combination of people) have to commit to that activity for two years. As Michael, Anthony, and others can attest, consideration of bug fixes isn't something you can just check out every few weeks. You get behind in a hell of a hurry and it's a major challenge to catch up. Right now, we are lucky to have one or two people for a relatively short period of time who serve as release managers for impending point releases. Andy> I think the solution is somewhat outside the core Python team's Andy> hands. Those of us who really care about stability need to get Andy> together and forge the consensus that we can skip some releases; It's more than that. You can "select for stability" now (sounds sort of like an aphorism you might hear in a management training course at GE). 2.0 is stable. Has been for quite awhile. It's not going anywhere. If you want to preserve it beyond when its available on the website, just mirror it. You could also choose 1.5.2. It's even more stable. However, I presume that if you want to skip some releases you'd also like some bugs fixed once in awhile. If not, then I don't see the problem with the status quo. Latch onto a release, tell your clients you require it to run your software, then revisit the issue every two years. Andy> In summary I'd like a note from Guido on the download page saying Andy> "we do releases every 6 months but if you prefer a longer cycle, Andy> go for 2.1.3 for now, and I will designate a stable 2.4.X release Andy> some time in 2003". It doesn't mean 2.2 or 2.3 is bad in any way, Andy> but it gives us a decent planning horizon. I don't understand how you can ask that of Guido. You can't ask that of Microsoft (well, you can, but you might not get an answer you like). Andy> Maybe EuroPython is the time to pick this up. I expect a full report afterwards. :-) Skip From tim.one@comcast.net Thu Apr 11 07:06:19 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 11 Apr 2002 02:06:19 -0400 Subject: [Python-Dev] Regression in unicodestr.encode()? In-Reply-To: Message-ID: [Tim] > ... > If you run Barry's test under a debug build, a call to pymalloc's > realloc complains immediately upon entry that the passed-in address > suffered overwrites ... This deserves emphasizing because the debug pymalloc is new: we've had two(!) memory corruption problems since this has been available, and the debug malloc was a real help both times. This particular case was a best case: the debug realloc detected the corruption almost immediately after the overwrite occurred, and called Py_FatalError() after printing some helpful clues. But note that the "serial number" it printed was insane: the block was made by call #1852047475 to debug malloc/realloc That's because the overwrite was *so* bad it corrupted bytes beyond the end of the 4 trailing "forbidden bytes", and that's where the serial number is stored by the debug pymalloc. We'll all be much happier if you stick to modest off-by-1 fatal errors in the future . The other case was the gc-versus-trashcan disaster. The debug pymalloc didn't catch the corruption directly, but, when things blew up, it was dead obvious in the debugger that gc was crawling over an already-free()ed object (the object fields were entirely filled with pymalloc's "dead byte" value, 0xdb, which the debug pymalloc free() sprays into the released memory block). So this is a powerful low-tech tool. If you want to become a wizard at it fast, deliberately provoke some object memory management errors in your source tree, and just play with what happens then in a debug build. From mwh@python.net Thu Apr 11 09:52:11 2002 From: mwh@python.net (Michael Hudson) Date: 11 Apr 2002 09:52:11 +0100 Subject: [Python-Dev] Re: Stability and change In-Reply-To: Aahz's message of "Wed, 10 Apr 2002 18:26:31 -0400" References: <15538.7428.436583.908407@12-248-41-177.client.attbi.com> <15539.49195.297507.616138@12-248-41-177.client.attbi.com> <15540.14010.181095.298392@slothrop.zope.com> <20020410143241.GD10792@panix.com> <200204101512.g3AFCXd22433@pcp742651pcs.reston01.va.comcast.net> <20020410152135.GB23597@panix.com> <200204102203.g3AM39L23344@pcp742651pcs.reston01.va.comcast.net> <20020410222631.GA4145@panix.com> Message-ID: <2mofgqr51g.fsf@starship.python.net> Aahz writes: > On Wed, Apr 10, 2002, Guido van Rossum wrote: > > > >> * As I've said in other posts, what we're really looking for isn't so > >> much people to download these releases as people to *use* these > >> releases. That means they need to at least feel comfortable > >> submitting bug reports and preferably also feel comfortable creating > >> context diffs and stuff like that. We want to create an environment > >> that eases people like me into this process. We're starting to go > >> in that direction with the Developer's Guide, but I want to keep > >> focused on it. > > > > Maybe in addition to a developer's guide we need a tester's guide, > > focusing on how to test new releases and report bugs. > > I'll make that part of the Developer's Guide? Or a separate > link off the main page? Here's some material for that page: + please include the options you passed to ./configure + please include precise platform details + if the build fails, please include a copy of the error messages (the entire output of "make" is overkill, but more is better than less. on sf attaching the output of make to the report is sensible) + if a test fails, please run it in verbose mode too, and include the output of that (& explain how to do this - I got a lot of copies of the output of "python -v test_math.py -v" after 221c1...) + an exhortation to run their application's regression suites using the new Python (I tried to get people to do this for 221, but I don't think anyone did. I still think it would be a good idea). Cheers, M. -- 93. When someone says "I want a programming language in which I need only say what I wish done," give him a lollipop. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html From mwh@python.net Thu Apr 11 10:20:06 2002 From: mwh@python.net (Michael Hudson) Date: 11 Apr 2002 10:20:06 +0100 Subject: [Python-Dev] Stability and Change In-Reply-To: Neil Schemenauer's message of "Wed, 10 Apr 2002 10:48:12 -0700" References: <200204100306.g3A36CE20294@pcp742651pcs.reston01.va.comcast.net> <15539.48208.510028.637885@12-248-41-177.client.attbi.com> <20020410104812.A1445@glacier.arctrix.com> Message-ID: <2mlmbur3qx.fsf@starship.python.net> Neil Schemenauer writes: > Based on resources available the Python project I think we are doing a > nearly optimal job. Me too. And I think we're getting better too, one way and another. Obviously, the situation isn't perfect, but I don't think that exaggerated changes are necessary either. Cheers, M. -- If you give someone Fortran, he has Fortran. If you give someone Lisp, he has any language he pleases. -- Guy L. Steele Jr, quoted by David Rush in comp.lang.scheme.scsh From mwh@python.net Thu Apr 11 10:30:32 2002 From: mwh@python.net (Michael Hudson) Date: 11 Apr 2002 10:30:32 +0100 Subject: [Python-Dev] RELEASED: Python 2.2.1 In-Reply-To: "Fred L. Drake, Jr."'s message of "Wed, 10 Apr 2002 15:13:44 -0400" References: <20020410105940.A13862@ActiveState.com> <15540.36584.430237.409927@grendel.zope.com> Message-ID: <2mit6yr39j.fsf@starship.python.net> "Fred L. Drake, Jr." writes: > Trent Mick writes: > > It might be nice to move the 'r221' CVS tag further along the > > 'release22-maint' branch. The tree at the current 'r221' location fails > > to build the docs with the following: > > > > make[1]: *** No rule to make target `whatsnew/whatsnew22.tex', needed by `html/whatsnew/whatsnew22.html'. Stop. > > > > 'whatsnew22.tex' was added after the tag. > > > > Thoughts? > > This should be fixed! I don't know who did the tagging, Me, obviously. I think I said when I did it here. > but don't remember any coordination activities. Well, only the 48 hour plan I posted to python-dev. Admittedly that didn't say when I would tag the tree. > This isn't the only file that was changed since whatsnew22.tex was > added. It seems I didn't run "cvs up -d" before tagging the tree or building the tarball, so Doc/whatsnew/ isn't in the tarball or under the tag. Sorry guys. At least nothing else got affected by this. Can some who knows how sort out the CVS mess? I can learn, I guess, but I'd rather not risk cocking it up. The other changes that were made in the process of building the windows installer and the docs I really don't care about (i.e. the changes Trent posted). Is this worth updating the tarball for? Cheers, M. -- Reading Slashdot can [...] often be worse than useless, especially to young and budding programmers: it can give you exactly the wrong idea about the technical issues it raises. -- http://www.cs.washington.edu/homes/klee/misc/slashdot.html#reasons From fdrake@acm.org Thu Apr 11 12:53:49 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 11 Apr 2002 07:53:49 -0400 Subject: [Python-Dev] RELEASED: Python 2.2.1 In-Reply-To: <2mit6yr39j.fsf@starship.python.net> References: <20020410105940.A13862@ActiveState.com> <15540.36584.430237.409927@grendel.zope.com> <2mit6yr39j.fsf@starship.python.net> Message-ID: <15541.31053.138403.162912@grendel.zope.com> Michael Hudson writes: > Me, obviously. I think I said when I did it here. You probably did, but there was a pretty mail volume earlier this week, so probably got lost in the pile. ;-) Heck, it might even still be waiting for me to get to it! > Well, only the 48 hour plan I posted to python-dev. Admittedly that > didn't say when I would tag the tree. I liked having the plan. Perhaps this could be added to the release procedure PEPs. > Can some who knows how sort out the CVS mess? I can learn, I guess, > but I'd rather not risk cocking it up. > > The other changes that were made in the process of building the > windows installer and the docs I really don't care about (i.e. the > changes Trent posted). > > Is this worth updating the tarball for? Avoiding this question is why we need the specific coordination on the tagging. ;-) Should the tarball define the release, or does the release define the tarball? Doesn't matter as long as they're the same. The value of the tag is that it reflects the release. Since the tarball went out broken (not matching the files used to build), I'm not sure what should be done to fix things. I'll be glad to handle the CVS machinery if the determination is that the tag should be moved. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From guido@python.org Thu Apr 11 13:45:07 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 11 Apr 2002 08:45:07 -0400 Subject: [Python-Dev] Re: Unittest list In-Reply-To: Your message of "Wed, 10 Apr 2002 22:51:50 EDT." <15540.64070.769894.589552@grendel.zope.com> References: <3CB46566.C9DC62B7@metaslash.com> <200204101907.g3AJ7ch22996@pcp742651pcs.reston01.va.comcast.net> <3CB4F03D.C584533C@metaslash.com> <15540.64070.769894.589552@grendel.zope.com> Message-ID: <200204111245.g3BCj7v25104@pcp742651pcs.reston01.va.comcast.net> > I'd guess that a lot of untested functions here are the slot wrappers, > which are macro generated and all boilerplate, except for a couple. > I'd be interested in seeing the list to see if my intuition is right > though; if wrong, it's time to add some tests. And it probably > wouldn't hurt to add tests for the boilerplate stuff anyway, just to > be extra careful about understanding expectations and protecting > against regression failures. Same for typeobject.c, which has two (!) different wrappers for most slots. After 2.2 came out (and also during beta testing) several bugs in these were discovered one at a time -- I clearly don't test them all. --Guido van Rossum (home page: http://www.python.org/~guido/) From gmcm@hypernet.com Thu Apr 11 13:48:08 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Thu, 11 Apr 2002 08:48:08 -0400 Subject: [Python-Dev] Stability and Change In-Reply-To: <15541.9033.825422.207087@12-248-41-177.client.attbi.com> References: Message-ID: <3CB54DC8.27532.6B8399AC@localhost> On 11 Apr 2002 at 0:46, Skip Montanaro wrote: > ... Guaranteeing the shelf-life of releases for > potentially two years, when PythonLabs typically > releases every six months, means you're going to > have four active releases going at any one time. > That means every bug fix that is applied to the main > CVS trunk has to at least be considered for > backporting to all those other releases, maybe not > applied, but at least considered. Practically > speaking, that means you need a release manager for > each of those active releases, and they have (or a > combination of people) have to commit to that > activity for two years. I think you are overstating by a bit. If a bug fix doesn't apply to release N, it (normally) won't apply to release N-1. The oldest release will see only a small fraction of the activity the most recent has. Of course, more work is still more work. -- Gordon > http://www.mcmillan-inc.com/ From guido@python.org Thu Apr 11 13:56:26 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 11 Apr 2002 08:56:26 -0400 Subject: [Python-Dev] Can the cgi module be made Unicode-aware? In-Reply-To: Your message of "Thu, 11 Apr 2002 00:02:40 CDT." <15541.6384.351664.886436@12-248-41-177.client.attbi.com> References: <15541.6384.351664.886436@12-248-41-177.client.attbi.com> Message-ID: <200204111256.g3BCuQk25210@pcp742651pcs.reston01.va.comcast.net> > I keep trying to handle various places in my code where I can get > input in non-ASCII encodings. Today I realized the cgi module does > nothing to translate Unicode data into unicode objects. I see in > one instance that I am getting data that is clearly utf-8 encoded, > but I see nothing in the CGI script's environment variables to > suggest the client web browser told the server how the data was > encoded other than the obvious "Content-Type: > application/x-www-form-urlencoded". Is utf-8 implied for the data > once the url encoding has been reversed? I very much doubt it. You probably received that UTF-8 data from a non-standard-conforming browser. > Should the cgi module be made Unicode-aware? If so, how? I can > never remember the incantation to convert non-ASCII string objects > to Unicode objects and nothing I've tried by trial-and-error so far > works. I must be misunderstanding your question, because the answer I'm thinking of is unicode(s,'utf8') and that can't possibly be what you can never remember. > I *don't* want to adopt the workaround outlined in FAQ > question 4.102 (change the default site-wide encoding). Perhaps > that question should be extended with more appropriate information > about converting raw strings with non-ASCII content to unicode. (There's also an approach that tries to compare the converted to the unconverted version and catches the exception; if no exception is raised, the input string was pure ASCII and the Unicode conversion is unnecessary.) --Guido van Rossum (home page: http://www.python.org/~guido/) From akuchlin@mems-exchange.org Thu Apr 11 14:04:08 2002 From: akuchlin@mems-exchange.org (akuchlin@mems-exchange.org) Date: Thu, 11 Apr 2002 09:04:08 -0400 Subject: [Python-Dev] Stability Metrics In-Reply-To: <003201c1e113$e1aca1e0$0cb53bd0@othello>; from python@rcn.com on Thu, Apr 11, 2002 at 12:46:43AM -0400 References: <20020411025313.29814.31653.Mailman@mail.python.org> <003201c1e113$e1aca1e0$0cb53bd0@othello> Message-ID: <20020411090408.B26178@mems-exchange.org> On Thu, Apr 11, 2002 at 12:46:43AM -0400, Raymond Hettinger wrote: >Whatever metric is choosen (and I DO think having a stability >metric is good), it should have a clear interpretation that >a higher number is good and a lower number is bad or vice-versa. The number itself is meaningless as a stability indicator; Skip's suggestion is that the number should drop with successive releases as the unfixed bugs get more and more obscure. So 2.2.1 fixes 140 bugs, and 2.2.2 fixes 70, and 2.2.3 fixes 15, and so on. --amk (www.amk.ca) Here are we -- and yonder yawns the universe. -- H.P. Lovecraft, in a letter to Frank Belknap Long, February 20 1929 From mwh@python.net Thu Apr 11 14:04:27 2002 From: mwh@python.net (Michael Hudson) Date: 11 Apr 2002 14:04:27 +0100 Subject: [Python-Dev] RELEASED: Python 2.2.1 In-Reply-To: "Fred L. Drake, Jr."'s message of "Thu, 11 Apr 2002 07:53:49 -0400" References: <20020410105940.A13862@ActiveState.com> <15540.36584.430237.409927@grendel.zope.com> <2mit6yr39j.fsf@starship.python.net> <15541.31053.138403.162912@grendel.zope.com> Message-ID: <2mpu16xu78.fsf@starship.python.net> "Fred L. Drake, Jr." writes: > Michael Hudson writes: > > Me, obviously. I think I said when I did it here. > > You probably did, but there was a pretty mail volume earlier this > week, so probably got lost in the pile. ;-) Heck, it might even > still be waiting for me to get to it! Quite likely. > > Well, only the 48 hour plan I posted to python-dev. Admittedly that > > didn't say when I would tag the tree. > > I liked having the plan. Perhaps this could be added to the release > procedure PEPs. Yeah. The PEP need to have an axe taken to it now that we ignore sf, too. > > Can some who knows how sort out the CVS mess? I can learn, I guess, > > but I'd rather not risk cocking it up. > > > > The other changes that were made in the process of building the > > windows installer and the docs I really don't care about (i.e. the > > changes Trent posted). > > > > Is this worth updating the tarball for? > > Avoiding this question is why we need the specific coordination on the > tagging. ;-) Perhaps. I'm not sure it would have helped, unless someone had said "you have run cvs up -d, haven't you?"[0]. > Should the tarball define the release, or does the release define > the tarball? Doesn't matter as long as they're the same. The value > of the tag is that it reflects the release. How many people build the docs from source? > Since the tarball went out broken (not matching the files used to > build), I'm not sure what should be done to fix things. I'll be glad > to handle the CVS machinery if the determination is that the tag > should be moved. I really think CVS should be fixed. It's not really a case of moving a tag, it's more a case of adding files to a tag. Cheers, M. [0] and WTF does cvs not pick up new directories by default? How is that a good idea? -- Never meddle in the affairs of NT. It is slow to boot and quick to crash. -- Stephen Harris -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html From guido@python.org Thu Apr 11 14:18:01 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 11 Apr 2002 09:18:01 -0400 Subject: [Python-Dev] RELEASED: Python 2.2.1 In-Reply-To: Your message of "Thu, 11 Apr 2002 07:53:49 EDT." <15541.31053.138403.162912@grendel.zope.com> References: <20020410105940.A13862@ActiveState.com> <15540.36584.430237.409927@grendel.zope.com> <2mit6yr39j.fsf@starship.python.net> <15541.31053.138403.162912@grendel.zope.com> Message-ID: <200204111318.g3BDI1M25287@pcp742651pcs.reston01.va.comcast.net> > > Can some who knows how sort out the CVS mess? I can learn, I guess, > > but I'd rather not risk cocking it up. To move a tag, simply retag with the same tag but adding the -F option. If you retag a whole tree with -F, you only get "T filename" lines for files where the tag is actually moved. In this case I'd say leave it. > > The other changes that were made in the process of building the > > windows installer and the docs I really don't care about (i.e. the > > changes Trent posted). > > > > Is this worth updating the tarball for? > > Avoiding this question is why we need the specific coordination on the > tagging. ;-) Should the tarball define the release, or does the > release define the tarball? Doesn't matter as long as they're the > same. The value of the tag is that it reflects the release. > > Since the tarball went out broken (not matching the files used to > build), I'm not sure what should be done to fix things. I'll be glad > to handle the CVS machinery if the determination is that the tag > should be moved. If we change either the tarball or the binary release (Win32 installer and RPMs) now, it would have to be a new release number, else there could be confusion (the MD5 checksum of the tarball would change, and who knows if there are people who already picked up the tarball and start redistributing, etc.). I think that's not worth it, so let this be a warning for all. Make a change to PEP 102 to remind you of what to do in the future. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Apr 11 14:25:40 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 11 Apr 2002 09:25:40 -0400 Subject: [Python-Dev] RELEASED: Python 2.2.1 In-Reply-To: Your message of "11 Apr 2002 14:04:27 BST." <2mpu16xu78.fsf@starship.python.net> References: <20020410105940.A13862@ActiveState.com> <15540.36584.430237.409927@grendel.zope.com> <2mit6yr39j.fsf@starship.python.net> <15541.31053.138403.162912@grendel.zope.com> <2mpu16xu78.fsf@starship.python.net> Message-ID: <200204111325.g3BDPec25381@pcp742651pcs.reston01.va.comcast.net> > I really think CVS should be fixed. It's not really a case of moving > a tag, it's more a case of adding files to a tag. No, you should be able to recreate the tarball by using the tag. Too bad that you can't exactly recreate the binaries -- a binary is what it is, and rarely duplicatable 100%. > [0] and WTF does cvs not pick up new directories by default? How is > that a good idea? Who knows. For years, I've got this in my .profile: alias sup='cvs -z3 -q update -d -P' It's the -d -P part that does it. --Guido van Rossum (home page: http://www.python.org/~guido/) From sjoerd@acm.org Thu Apr 11 14:43:44 2002 From: sjoerd@acm.org (Sjoerd Mullender) Date: Thu, 11 Apr 2002 15:43:44 +0200 Subject: [Python-Dev] RELEASED: Python 2.2.1 References: <20020410105940.A13862@ActiveState.com> <15540.36584.430237.409927@grendel.zope.com> <2mit6yr39j.fsf@starship.python.net> <15541.31053.138403.162912@grendel.zope.com> <2mpu16xu78.fsf@starship.python.net> <200204111325.g3BDPec25381@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CB59310.2588CB44@acm.org> Guido van Rossum wrote: > > > I really think CVS should be fixed. It's not really a case of moving > > a tag, it's more a case of adding files to a tag. > > No, you should be able to recreate the tarball by using the tag. Too > bad that you can't exactly recreate the binaries -- a binary is what > it is, and rarely duplicatable 100%. > > > [0] and WTF does cvs not pick up new directories by default? How is > > that a good idea? > > Who knows. For years, I've got this in my .profile: > > alias sup='cvs -z3 -q update -d -P' > > It's the -d -P part that does it. Instead you could have a file .cvsrc in your home directory with the following content: cvs -z3 update -dP This will use -z3 compression on all CVS commands (not just update, but also checkout, diff and commit). Also it will work for any which way you invoke CVS, not just when you do it from a shell. From neal@metaslash.com Thu Apr 11 14:44:24 2002 From: neal@metaslash.com (Neal Norwitz) Date: Thu, 11 Apr 2002 09:44:24 -0400 Subject: [Python-Dev] RELEASED: Python 2.2.1 References: <20020410105940.A13862@ActiveState.com> <15540.36584.430237.409927@grendel.zope.com> <2mit6yr39j.fsf@starship.python.net> <15541.31053.138403.162912@grendel.zope.com> <2mpu16xu78.fsf@starship.python.net> <200204111325.g3BDPec25381@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CB59338.FAEE78D7@metaslash.com> Guido van Rossum wrote: > > > I really think CVS should be fixed. It's not really a case of moving > > a tag, it's more a case of adding files to a tag. > > No, you should be able to recreate the tarball by using the tag. Too > bad that you can't exactly recreate the binaries -- a binary is what > it is, and rarely duplicatable 100%. > > > [0] and WTF does cvs not pick up new directories by default? How is > > that a good idea? > > Who knows. For years, I've got this in my .profile: > > alias sup='cvs -z3 -q update -d -P' > > It's the -d -P part that does it. Or add a .cvsrc file to your home directory: update -dP diff -wu cvs -q ==== pass -dP options to cvs update (add directories, prune empty dirs) pass -wu options to cvs diff (ignore whitespace, unified diff) pass -q option to all cvs commands (ignore useless msgs about directories) Neal From jacobs@penguin.theopalgroup.com Thu Apr 11 14:46:16 2002 From: jacobs@penguin.theopalgroup.com (Kevin Jacobs) Date: Thu, 11 Apr 2002 09:46:16 -0400 (EDT) Subject: [Python-Dev] RELEASED: Python 2.2.1 In-Reply-To: <200204111325.g3BDPec25381@pcp742651pcs.reston01.va.comcast.net> Message-ID: On Thu, 11 Apr 2002, Guido van Rossum wrote: > Who knows. For years, I've got this in my .profile: > > alias sup='cvs -z3 -q update -d -P' > > It's the -d -P part that does it. A slick way of doing this is to create a ~/.cvsrc file, which allows you to add default arguments. Here is mine: cvs -q update -d -P diff -u The first line adds '-q' to all cvs commands by default. I work with large projects with many directories and a global '-q' really cleans up the output. The second two lines automatically add -d -P and -u to cvs update and cvs diff respectively. -Kevin -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com From guido@python.org Thu Apr 11 15:10:51 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 11 Apr 2002 10:10:51 -0400 Subject: [Python-Dev] Stability Metrics [was Stability and Change] In-Reply-To: Your message of "Thu, 11 Apr 2002 00:46:43 EDT." <003201c1e113$e1aca1e0$0cb53bd0@othello> References: <20020411025313.29814.31653.Mailman@mail.python.org> <003201c1e113$e1aca1e0$0cb53bd0@othello> Message-ID: <200204111410.g3BEApL25642@pcp742651pcs.reston01.va.comcast.net> [AMK] > > I've done this ever since whatsnew21. My numbers, surely > > underestimates, are: > > > > 2.1: 117 patches, 136 bugs > > 2.2: 527 patches, 683 bugs > > 2.2.1: 139 patches, 143 bugs [RH] > Whoa there! The conversation is now trending toward bugs > as a metric of merit rather than demerit. Of course, fixing > bugs is good, but not having them in the first place is better. > > Before choosing a publicity metric, consider the significance > of the metric if it is very large or very small. In the above > example, 0 bugs would likely indicate that no maintenance > is taking place. Having 10,000 bugs would indicate that > the release process was a disaster. > > Whatever metric is choosen (and I DO think having a stability > metric is good), it should have a clear interpretation that > a higher number is good and a lower number is bad or vice-versa. I think it's nice to keep track of this for ourselves (despite the interpretational problems). But I'm strengthening my position about the use of statistics in communicating to the users: I think that would be bad. Statistics can easily be misused, misinterpreted, and countered. --Guido van Rossum (home page: http://www.python.org/~guido/) From mwh@python.net Thu Apr 11 15:09:48 2002 From: mwh@python.net (Michael Hudson) Date: 11 Apr 2002 15:09:48 +0100 Subject: [Python-Dev] RELEASED: Python 2.2.1 In-Reply-To: Neal Norwitz's message of "Thu, 11 Apr 2002 09:44:24 -0400" References: <20020410105940.A13862@ActiveState.com> <15540.36584.430237.409927@grendel.zope.com> <2mit6yr39j.fsf@starship.python.net> <15541.31053.138403.162912@grendel.zope.com> <2mpu16xu78.fsf@starship.python.net> <200204111325.g3BDPec25381@pcp742651pcs.reston01.va.comcast.net> <3CB59338.FAEE78D7@metaslash.com> Message-ID: <2mhemis4wj.fsf@starship.python.net> Neal Norwitz writes: > Guido van Rossum wrote: > > > > > I really think CVS should be fixed. It's not really a case of moving > > > a tag, it's more a case of adding files to a tag. > > > > No, you should be able to recreate the tarball by using the tag. Too > > bad that you can't exactly recreate the binaries -- a binary is what > > it is, and rarely duplicatable 100%. > > > > > [0] and WTF does cvs not pick up new directories by default? How is > > > that a good idea? > > > > Who knows. For years, I've got this in my .profile: > > > > alias sup='cvs -z3 -q update -d -P' > > > > It's the -d -P part that does it. > > Or add a .cvsrc file to your home directory: > > update -dP Hmm. It seems that up -dP in your .cvsrc *doesn't* do what you might think... Cheers, M. -- [1] If you're lost in the woods, just bury some fibre in the ground carrying data. Fairly soon a JCB will be along to cut it for you - follow the JCB back to civilsation/hitch a lift. -- Simon Burr, cam.misc From barry@zope.com Thu Apr 11 15:15:18 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 11 Apr 2002 10:15:18 -0400 Subject: [Python-Dev] Can the cgi module be made Unicode-aware? References: <15541.6384.351664.886436@12-248-41-177.client.attbi.com> Message-ID: <15541.39542.449397.709483@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: SM> I keep trying to handle various places in my code where I can SM> get input in non-ASCII encodings. Today I realized the cgi SM> module does nothing to translate Unicode data into unicode SM> objects. I see in one instance that I am getting data that is SM> clearly utf-8 encoded, but I see nothing in the CGI script's SM> environment variables to suggest the client web browser told SM> the server how the data was encoded other than the obvious SM> "Content-Type: application/x-www-form-urlencoded". Is utf-8 SM> implied for the data once the url encoding has been reversed? If the data is encoded in a non-ascii character set, wouldn't (or shouldn't) the Content-Type: header include a charset="xxx" parameter? -Barry From neal@metaslash.com Thu Apr 11 15:19:21 2002 From: neal@metaslash.com (Neal Norwitz) Date: Thu, 11 Apr 2002 10:19:21 -0400 Subject: [Python-Dev] RELEASED: Python 2.2.1 References: <20020410105940.A13862@ActiveState.com> <15540.36584.430237.409927@grendel.zope.com> <2mit6yr39j.fsf@starship.python.net> <15541.31053.138403.162912@grendel.zope.com> <2mpu16xu78.fsf@starship.python.net> <200204111325.g3BDPec25381@pcp742651pcs.reston01.va.comcast.net> <3CB59338.FAEE78D7@metaslash.com> <2mhemis4wj.fsf@starship.python.net> Message-ID: <3CB59B69.A9E3965F@metaslash.com> Michael Hudson wrote: > > > alias sup='cvs -z3 -q update -d -P' > > > > > > It's the -d -P part that does it. > > > > Or add a .cvsrc file to your home directory: > > > > update -dP > > Hmm. It seems that > > up -dP > > in your .cvsrc *doesn't* do what you might think... It's worked for me for years, AFAIK. What do you think it does? Are you suggesting it should be -d -P? Neal From mwh@python.net Thu Apr 11 15:23:28 2002 From: mwh@python.net (Michael Hudson) Date: 11 Apr 2002 15:23:28 +0100 Subject: [Python-Dev] RELEASED: Python 2.2.1 In-Reply-To: Neal Norwitz's message of "Thu, 11 Apr 2002 10:19:21 -0400" References: <20020410105940.A13862@ActiveState.com> <15540.36584.430237.409927@grendel.zope.com> <2mit6yr39j.fsf@starship.python.net> <15541.31053.138403.162912@grendel.zope.com> <2mpu16xu78.fsf@starship.python.net> <200204111325.g3BDPec25381@pcp742651pcs.reston01.va.comcast.net> <3CB59338.FAEE78D7@metaslash.com> <2mhemis4wj.fsf@starship.python.net> <3CB59B69.A9E3965F@metaslash.com> Message-ID: <2m662yz53z.fsf@starship.python.net> Neal Norwitz writes: > Michael Hudson wrote: > > > > > alias sup='cvs -z3 -q update -d -P' > > > > > > > > It's the -d -P part that does it. > > > > > > Or add a .cvsrc file to your home directory: > > > > > > update -dP > > > > Hmm. It seems that > > > > up -dP > > > > in your .cvsrc *doesn't* do what you might think... > > It's worked for me for years, AFAIK. What do you think it does? > Are you suggesting it should be -d -P? I dunno. But it turns out I already had "up -dP" in my .cvsrc, and you can be sure I ran "cvs up" several times before building the 221 tarball. Whether it's the up/update thing or the -dP/-d -P thing, I don't know. Cheers, M. -- ... so the notion that it is meaningful to pass pointers to memory objects into which any random function may write random values without having a clue where they point, has _not_ been debunked as the sheer idiocy it really is. -- Erik Naggum, comp.lang.lisp From skip@pobox.com Thu Apr 11 15:29:21 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 11 Apr 2002 09:29:21 -0500 Subject: [Python-Dev] Can the cgi module be made Unicode-aware? In-Reply-To: <200204111256.g3BCuQk25210@pcp742651pcs.reston01.va.comcast.net> References: <15541.6384.351664.886436@12-248-41-177.client.attbi.com> <200204111256.g3BCuQk25210@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15541.40385.925752.353594@12-248-41-177.client.attbi.com> >> I keep trying to handle various places in my code where I can get >> input in non-ASCII encodings. Today I realized the cgi module does >> nothing to translate Unicode data into unicode objects. I see in one >> instance that I am getting data that is clearly utf-8 encoded, but I >> see nothing in the CGI script's environment variables to suggest the >> client web browser told the server how the data was encoded other >> than the obvious "Content-Type: application/x-www-form-urlencoded". >> Is utf-8 implied for the data once the url encoding has been >> reversed? Guido> I very much doubt it. You probably received that UTF-8 data from Guido> a non-standard-conforming browser. I did some reading before nodding off last night. The
tag takes an optional "accept-charset" attribute, which can be a list. By default, the charset is "UNKNOWN", which is taken to commonly imply that the charset of the returned data is the same as the charset of the HTML page containing the form. Guido> I must be misunderstanding your question, because the answer I'm Guido> thinking of is unicode(s,'utf8') and that can't possibly be what Guido> you can never remember. I eventually did figure it out. :-) What I always forget is the stinking .encode() method to get it back to something printable. In my little dummy script I had print unicode(info, "utf-8") instead of print unicode(info, "utf-8").encode("some-encoding") It kept raising UnicodeError. I thought it was on the conversion to Unicode, but it was on the implicit conversion back to a printable string. The tracebacks look similar: Traceback (most recent call last): File "/home/skip/tmp/junk.py", line 3, in ? x = unicode(info) UnicodeError: ASCII decoding error: ordinal not in range(128) vs. Traceback (most recent call last): File "/home/skip/tmp/junk.py", line 4, in ? print x UnicodeError: ASCII encoding error: ordinal not in range(128) I was just missing (or misinterpreting) the words "decoding" and "encoding". Guido> (There's also an approach that tries to compare the converted to Guido> the unconverted version and catches the exception; if no Guido> exception is raised, the input string was pure ASCII and the Guido> Unicode conversion is unnecessary.) Yes, I use this technique elsewhere. Now, back to my original problem... :-) As far as I can tell, the underlying data encoding of the form's data is generally going to be implicit. Adding an "accept-charset" attribute to the does appear to have some effect on Content-Type in some instances, but not in all. I wrote a page with Latin-1 as the charset and specified utf-8 as the charset for the form. Upon submission, Opera added a charset attribute to the Content-Type header, Mozilla didn't. If I leave off accept-charset for the form, neither browser adds a charset attribute to the Content-Type header. In all cases I tried, both properly encoded the form data though. Can someone with access to Internet Explorer please give http://manatee.mojam.com/~skip/sample_form.html a try? Does it honor the charset attribute of the form (which is currently utf-8)? Does it add a charset to the Content-type header or not? The cgi programmer can't rely on charset information coming from the browser and will need a way to tell the cgi module what the charset of the incoming data is. I think FieldStorage and MiniFieldStorage need optional charset parameters and I think the charset needs to be used from the Content-Type header, if present. If neither are given, I think the current behavior should be retained (no interpretation/conversion of input data). After a bit of reflection, I'm not so sure I want to mess with cgi.py. :-) I'll try forcing my desired charset in my forms for the time being and see what happens. Maybe I'll fiddle around with a FieldStorage subclass, but that will be outside of cgi.py. I will update FAQ 4.102, however. Skip From fdrake@acm.org Thu Apr 11 15:33:20 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 11 Apr 2002 10:33:20 -0400 Subject: [Python-Dev] PEPs 101 & 102: release procedures Message-ID: <15541.40624.881972.563707@grendel.zope.com> I've updated PEP 101 and PEP 102 to remove the instructions relating to the SourceForge file release mechanism. If our contingent of experienced release managers could review the changes, I'd appreciate it! -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From barry@zope.com Thu Apr 11 15:37:51 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 11 Apr 2002 10:37:51 -0400 Subject: [Python-Dev] Can the cgi module be made Unicode-aware? References: <15541.6384.351664.886436@12-248-41-177.client.attbi.com> <200204111256.g3BCuQk25210@pcp742651pcs.reston01.va.comcast.net> <15541.40385.925752.353594@12-248-41-177.client.attbi.com> Message-ID: <15541.40895.883337.870236@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: SM> The cgi programmer can't rely on charset information coming SM> from the browser and will need a way to tell the cgi module SM> what the charset of the incoming data is. If you know. You might know what you accept, but you might not know what charset the data is coming back in. E.g. the form that specifies a message footer in Mailman 2.1 for a list that accepts 10 different languages. I guess you'd have to have some drop box that specified the language the footer text was in, and then know the mapping from that language to its charset. SM> After a bit of reflection, I'm not so sure I want to mess with SM> cgi.py. :-) Wise man! cgi.py is one of those modules that sucks just less than the amount of effort needed to rewrite it cleanly. -Barry From neal@metaslash.com Thu Apr 11 15:48:36 2002 From: neal@metaslash.com (Neal Norwitz) Date: Thu, 11 Apr 2002 10:48:36 -0400 Subject: [Python-Dev] RELEASED: Python 2.2.1 References: <20020410105940.A13862@ActiveState.com> <15540.36584.430237.409927@grendel.zope.com> <2mit6yr39j.fsf@starship.python.net> <15541.31053.138403.162912@grendel.zope.com> <2mpu16xu78.fsf@starship.python.net> <200204111325.g3BDPec25381@pcp742651pcs.reston01.va.comcast.net> <3CB59338.FAEE78D7@metaslash.com> <2mhemis4wj.fsf@starship.python.net> <3CB59B69.A9E3965F@metaslash.com> <2m662yz53z.fsf@starship.python.net> Message-ID: <3CB5A244.1F987320@metaslash.com> Michael Hudson wrote: > > Neal Norwitz writes: > > > Michael Hudson wrote: > > > > > Hmm. It seems that > > > > > > up -dP > > > > > > in your .cvsrc *doesn't* do what you might think... > > > > It's worked for me for years, AFAIK. What do you think it does? > > Are you suggesting it should be -d -P? > > I dunno. But it turns out I already had "up -dP" in my .cvsrc, and > you can be sure I ran "cvs up" several times before building the 221 > tarball. > > Whether it's the up/update thing or the -dP/-d -P thing, I don't know. I just tested, it appears you need the long name in .cvsrc. Even though the command line works just fine with up, up didn't work in .cvsrc. Neal From skip@pobox.com Thu Apr 11 15:51:47 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 11 Apr 2002 09:51:47 -0500 Subject: [Python-Dev] FAQ 4.102 has been updated - can someone proofread it? Message-ID: <15541.41731.671262.496339@12-248-41-177.client.attbi.com> Being the Unicode novice that I am, I'd feel better if someone proofread FAQ 4.102: http://www.python.org/cgi-bin/faqw.py?req=show&file=faq04.102.htp Much appreciated. Skip From vmilitaru@sympatico.ca Thu Apr 11 15:44:06 2002 From: vmilitaru@sympatico.ca (vmilitaru@sympatico.ca) Date: Thu, 11 Apr 2002 09:44:06 -0500 Subject: [Python-Dev] Java-ing Python Message-ID: <20020411094405.A409@eye.neb> Greetings, For a couple of weeks now, I've been trying to figure out how to translate some java code to python (reason: my app is in Zope, and I want to add some functionality which is coded in java ...re-coding from scratch is last resort). My 'quest' brought me to ANTLR, which eases such source-to-source translations. And this morning I 'discovered' python's Grammar file. And then lightning (at least it felt like it ...) stroke: what if I replace Python's Grammar file with a Java grammar (for example, the java grammar included in the ANTLR distribution). Hacked appropriately to map java source to the tokens defined in python's Grammar would in theory make python 'understand' java. In theory! I have no idea yet if such exact 1-to-1 mapping of tokens is at all possible. Moreover, replacing Python's Grammar file with a Java Grammar will obviously render any python source totally foreign to the python parser. Of course, the reason to brutally replacing Python's original grammar is to provide a fast hack who could fall into place and not break anything. The costlier alternative seems to hack more heavily the graminit.h and so on and add the Java Grammar 'alongside' Python's Grammar. I would like some feedback on this from the list, and would also like to know if Python provides some easier way to add a Grammar to the language, without the need to hack the C source ('cause, incidently, I kinda suck at C). Vio From guido@python.org Thu Apr 11 16:22:40 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 11 Apr 2002 11:22:40 -0400 Subject: [Python-Dev] FAQ 4.102 has been updated - can someone proofread it? In-Reply-To: Your message of "Thu, 11 Apr 2002 09:51:47 CDT." <15541.41731.671262.496339@12-248-41-177.client.attbi.com> References: <15541.41731.671262.496339@12-248-41-177.client.attbi.com> Message-ID: <200204111522.g3BFMes24367@odiug.zope.com> > Being the Unicode novice that I am, I'd feel better if someone proofread FAQ > 4.102: > > http://www.python.org/cgi-bin/faqw.py?req=show&file=faq04.102.htp > > Much appreciated. Hm, I think you've got the ordering wrong. Suggestion #1 would be to fix the application to do explicit encoding/decoding. Suggestion #2 would be to change the site default encoding by adding a sitecustomize.py. I'd leave direct hacking of site.py out of the suggestions. --Guido van Rossum (home page: http://www.python.org/~guido/) From vmilitaru@sympatico.ca Thu Apr 11 16:39:49 2002 From: vmilitaru@sympatico.ca (vmilitaru@sympatico.ca) Date: Thu, 11 Apr 2002 10:39:49 -0500 Subject: [Python-Dev] Java-ing Python /2 Message-ID: <20020411103948.A481@eye.neb> On second thought, making python parser understand java source is not as interesting as first thought, because what I am really looking for above anything else is to have python 'source'. With python understanding java, I am still stuck with the java source. So a source-to-source java-to-python translator tool is still the only way to go for me. Still, in my opinion it would be quite an interesting feature if python could make the addition of other grammars as easily as adding a new python module. A new grammar module would basically be a map of the foreign language to the python tokens. And a mechanism to deal with non-mappable features of the foreign language. Vio PS. If anyone knows of a tool to automate java-to-py source translation, do share your links, please. In the mean time, I'll attempt to do just that in ANTLR. From mal@lemburg.com Thu Apr 11 17:00:18 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 11 Apr 2002 18:00:18 +0200 Subject: [Python-Dev] Deprecating whrandom; and deprecating in general References: <3CB3093C.B7A22727@metaslash.com> <20020409210943.GB16531@panix.com> <3CB3F79D.A143D1C2@lemburg.com> <20020410142824.GC10792@panix.com> <3CB45EE3.E01DD8CE@lemburg.com> <15540.26640.454643.389710@beluga.mojam.com> <3CB47AEB.65B12B3E@lemburg.com> <20020410194920.GA18150@panix.com> <200204102200.g3AM0iQ23326@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CB5B312.47148FEC@lemburg.com> Guido van Rossum wrote: > > [Note changed subject] Thanks for taking time to answer my concerns. > [MAL] > > > You won't have much luck in doing s/whrandom/random/g on a > > > hard-copy Python text book and this is what Python newbies > > > read. I'm not even talking about potential Python users who > > > haven't gotten the slightest idea what sed is... :-) > > [Aahz] > > What Python textbooks use whrandom? > > > > > Note that the key point is a different one: every single > > > deprecation causes work (convincing management, code changes, > > > tests, deployment of the new code, training application > > > developers, writing/buying new books). Work costs money. > > > Money causes all sorts of problems. > > > > Would it be okay with you for whrandom to emit a deprecation warning, > > but not get deleted until Python 3.0? > > > > From my POV, what you're missing in this debate is the cost of > > keeping the code. If code exists, people ask questions about it. > > Random issues keep popping up on c.l.py and the simpler the > > situation, the easier to explain it. I am fine with people asking questions about it. In fact, for my customers I will probably create a special package which contains all the deprecated and/or removed code (without the deprecation warnings) -- perhaps we should augment the deprecation process to provide such a package as separate Python download to everybody ?! As an example for the reasoning here: Some existing scripts still use e.g. regex which is deprecated and probably will get removed from the core distribution soon. We can't update that code to use re because the script will have to expose the exact same semantics as regex provides and I don't know how to do this reliably with re (pcre or SRE). Note that these scripts are usually tools which again are used by other applications, so changing semantics is not an option either. It is acceptable to tell sys-admins of our customers to install the one extra package when they update Python, it is not acceptable to require them to change existing code and it's a maintenance nightmare to provide them with updated software since it is not actively being maintained anymore or was specifically written for the customer. > [PEP 4 comments] > > Now we need a rationale. AFAICT the rationale for deprecating > whrandom is that its functionality is a duplicate of the random > module, we don't want to have to maintain (and answer questions and > bug reports about) both modules, and the name is "wrong" (it refers to > a specific random algorithm which is known to be rather weak). That's > not a particularly strong rationale: there are only three very old SF > bug reports about it, and the revision history has been quiet since > January 2001. All changes after July 1998 seem to have been cosmetic, > from whitespace normalization to a docstring sweep (this can be used > as an argument pro or con deprecation). Groups.google.com shows > plenty of recent mention of it in c.l.py (a couple per month), so it > doesn't appear that nobody is using it. > > Suppose we decide to deprecate whrandom. The first opportunity to do > so will be in 2.3. (I think we shouldn't deprecate modules in micro > releases; the PEP is silent about that but I think it would be a bad > idea.) Then it will have to continue to exist for at least a year. > Because whrandom is still in active use (see above), I think the one > year deprecation period is too short in this case and we should use > two years. Maybe the warning should only be introduced in the second > year? That probably means (assuming releases every 6 months): > > 2.3 - deprecate in docs > > 2.5 - issue warning > > 2.7 - move to Lib/lib-old Perhaps we could make Lib/lib-old a separate package ?! > Now on to Marc's argument against deprecation: > > > > If not absolutely needed in order to make way for new techniques > > > or features, we should leave the old established code in place > > > rather than causing endless support sessions with customers > > > who fear that the code you've written for them 6 months ago > > > starts making not so funny noises or that drag you to update > > > the code on a warranty basis. > > > > > > What's worse, is that management will not be delighted to > > > see that this "new technology we use, called Python" causes > > > quality assurance and training costs every 8-12 months. I > > > wouldn't want Python to go down that road, now that the > > > ball is starting to roll. > > I think there are two different typical use cases for Python code, and > they have different preferences for deprecation (and other) warnings. > > Use case #1 is what we usually think of. Some programmer is > maintaining some Python code. When he upgrades the Python version he > is using, being the programmer he probably wants to find out about > deprecated features in the new Python version, so that he can fix his > code so that it won't break outright in the future. IOW, this > programmer is glad that there are deprecation warnings (compared to > the alternative, where he had no idea that he was using a deprecated > feature until it was removed and his program broke outright). True; however, deprecation warnings are run-time messages and not every module in a package is always used by the programmer, so the warnings may appear after a few months down the road. I would much rather like a tool like PyChecker scan *all* the code for deprecated modules and maybe a similar tool for the C API. > Use case #2 is Marc-Andre's and Fredrik's situation: a company > writes a program in Python which runs at a customer's site. Assume > the program is not large and self-contained enough to warrant it > coming with its own version of Python. The customer has lots of > Python code in production, maybe some that they wrote themselves, some > that they bought from different places, and maybe even some open > source downloads. > > At some point the customer decides to upgrade their Python -- maybe > because they like to be on the bleeding edge for their own code base, > maybe because they need this for a new open source download, maybe > they believe it will fix some Python problem they've experienced > intermittently. Now the program they brought from PythonWare or from > eGEnix suddenly starts to emit a warning each time it is run. > > This is annoying: maybe the warning goes to a log file where > unexpected things cause alarms to go off, maybe it shows up in an end > user's shell window where it causes confusion, maybe there are a lot > of warnings and the extra output is simply annoying. These users > probably aren't Python savvy, so they may panic and call customer > service. Understandably, nobody is happy. Marc-Andre and Fredrik > already know their program issues a warning, and they've fixed it in > their own version, and they don't want to waste time telling every > user how to disable or ignore the warning, or shipping upgrades. It not so much about wasting time, it's about bad PR: to the customer it appears that you've written malfunctioning code. > How do we satisfy both use cases? We could turn off deprecation > warnings by default. But I suspect that most programmers in use case > #1 would probably never turn on deprecation warnings, thereby > defeating the point of them. > > I would like to suggest that instead, professional software > distributors like eGenix and PythonWare tweak their distributions to > turn of warnings by default. This is easy enough: in the main > program, insert > > import warnings > warnings.filterwarnings("ignore") > > at the top (before other imports). If the code needs to run on Python > 2.0 or before, insert the whole thing inside a try: / except > ImportError: pass. If you want an easy way to enable warnings, you > can invent something else (e.g. check an environment variable). > > Would this be acceptable? Yes. I would still like the option to continue using code which was deprecated by PythonLabs. Just because code is unmaintained doesn't mean that it cannot be used anymore. So how about this: * deprecated modules get moved into a separate distutils based package and are made available on python.org together with the new Python version; the package should contain a copy of PEP 4 for the curious reader * deprecated C APIs are grouped in a file pyold.h which only gets included by Python.h if Py_DEPRECATED is defined; that header files should contain a list similar to PEP 4 for the deprecated C APIs and if possible include a description of how to update existing code Both techniques aim for stability: they make the deprecation process more visible and they provide downgrade paths for existing installed code. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From martin@v.loewis.de Thu Apr 11 17:19:47 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 11 Apr 2002 18:19:47 +0200 Subject: [Python-Dev] Can the cgi module be made Unicode-aware? In-Reply-To: <200204111256.g3BCuQk25210@pcp742651pcs.reston01.va.comcast.net> References: <15541.6384.351664.886436@12-248-41-177.client.attbi.com> <200204111256.g3BCuQk25210@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > > "Content-Type: > > application/x-www-form-urlencoded". Is utf-8 implied for the data > > once the url encoding has been reversed? > > I very much doubt it. You probably received that UTF-8 data from a > non-standard-conforming browser. That's partially a bug in HTTP forms, partially a bug in the browsers, and partially a bug in many CGI scripts. The original URL encoding of form paramters (in the URL itself, using GET) does not allow a specification of the encoding; that's the bug in HTTP. To work around this, *all* browsers (by silent convention) send form parameters in the encoding that the document was in. So if the document containing the form is in UTF-8, they will send the form parameters in UTF-8. Of course, unless you *know* what encoding the original document had, there is no way of telling that it is UTF-8. The RFC specifies that, if application/x-www-form-urlencoded is used, text fields *should* have a Content-Type field, with a charset argument. The bug in the browsers is that they omit the Content-Type declaration for individual fields. I've reported this bug for MSIE, Mozilla, and Opera. Some Mozilla author told me that they tried sending a charset= parameter, and that many Web sites broke when this is done - this is the bug in many CGI scripts. Regards, Martin From martin@v.loewis.de Thu Apr 11 17:26:55 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 11 Apr 2002 18:26:55 +0200 Subject: [Python-Dev] Can the cgi module be made Unicode-aware? In-Reply-To: <15541.40385.925752.353594@12-248-41-177.client.attbi.com> References: <15541.6384.351664.886436@12-248-41-177.client.attbi.com> <200204111256.g3BCuQk25210@pcp742651pcs.reston01.va.comcast.net> <15541.40385.925752.353594@12-248-41-177.client.attbi.com> Message-ID: Skip Montanaro writes: > I did some reading before nodding off last night. The tag takes an > optional "accept-charset" attribute, which can be a list. No, it doesn't - that's a proprietary extension. Or, maybe I'm missing something: where did you find a statement that this is "official" in any sense? > As far as I can tell, the underlying data encoding of the form's data is > generally going to be implicit. Unfortunately. RFC 1867 specifies that browsers should use a Content-Type in a multipart/form-data message, but none of the current browsers does. > Adding an "accept-charset" attribute to the does appear to > have some effect on Content-Type in some instances, but not in all. It might depend on the browser, since it's proprietary. > The cgi programmer can't rely on charset information coming from the browser > and will need a way to tell the cgi module what the charset of the incoming > data is. I think FieldStorage and MiniFieldStorage need optional charset > parameters and I think the charset needs to be used from the Content-Type > header, if present. Of course, if you also have uploaded files, this cannot work: the file data never follow the encoding - only the "text" fields do. Regards, Martin From mal@lemburg.com Thu Apr 11 17:30:34 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 11 Apr 2002 18:30:34 +0200 Subject: [Python-Dev] rotor module (exporting encryption) References: <3C95C439.D5B3C573@lemburg.com> <3C961E37.56BB3F30@metaslash.com> <3C962276.319950D5@lemburg.com> <3C966AEA.CA91701C@metaslash.com> <3C96FDA9.95F0750C@lemburg.com> <3CA0DA9E.49460419@metaslash.com> Message-ID: <3CB5BA2A.D3957A8B@lemburg.com> Neal Norwitz wrote: > > I spoke with Lynn Griffin at the BXA about this subject. > She said the only requirement is to fill out this form (I filled > in some of the details): > > BEGIN FORM > ---------- > SUBJECT: ENC NOTIFICATION > > SUBMISSION TYPE: ENC > SUBMITTED BY: > SUBMITTED FOR: (company or person exporting the encryption item) > POINT OF CONTACT: > PHONE and/or FAX: > MANUFACTURER: (if relevant) > PRODUCT NAME/MODEL #: Python 2.2 > ECCN: 5D002 > NOTIFICATION: http://www.python.org > http://python.org/ftp/python/2.2/ > http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Modules/rotormodule.c > -------- > END FORM > > The instructions are available from: http://www.bxa.doc.gov/Encryption/PubAvailEncSourceCodeNofify.html. > > The email should be sent to: crypt@bxa.doc.gov. > > She also recommended a disclaimer on the website that the software > is exported under US Export laws and can't be exported to the 7-8 countries > or other specifically prohibited individuals. This part is interesting; we should add a note to all download pages on python.org (and also to the CVS checkout page). It's silly, but I guess we don't have a choice. > It seems we should submit this form, but for whom? PSF? > Who should be the point of contact? The PSF secretary (Jeremy) will submit the form. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From guido@python.org Thu Apr 11 17:43:51 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 11 Apr 2002 12:43:51 -0400 Subject: [Python-Dev] Deprecating whrandom; and deprecating in general In-Reply-To: Your message of "Thu, 11 Apr 2002 18:00:18 +0200." <3CB5B312.47148FEC@lemburg.com> References: <3CB3093C.B7A22727@metaslash.com> <20020409210943.GB16531@panix.com> <3CB3F79D.A143D1C2@lemburg.com> <20020410142824.GC10792@panix.com> <3CB45EE3.E01DD8CE@lemburg.com> <15540.26640.454643.389710@beluga.mojam.com> <3CB47AEB.65B12B3E@lemburg.com> <20020410194920.GA18150@panix.com> <200204102200.g3AM0iQ23326@pcp742651pcs.reston01.va.comcast.net> <3CB5B312.47148FEC@lemburg.com> Message-ID: <200204111643.g3BGhqN25089@odiug.zope.com> [MAL] > Some existing scripts still use e.g. regex which is deprecated > and probably will get removed from the core distribution soon. > We can't update that code to use re because the script will have > to expose the exact same semantics as regex provides and I don't > know how to do this reliably with re (pcre or SRE). Note that > these scripts are usually tools which again are used by other > applications, so changing semantics is not an option either. I'm not sure I understand. Are these other packages generating regular expressions in the regex syntax that your script then applies somewhere? This is indeed a problem -- unless you can keep your customers on Python <= 2.2 forever, eventually the regex module will die of old age. Perhaps what's necessary is a translation module that translates the regex syntax into re syntax? Such a module could be provided as part of future releases as regex, without deprecation (at least until you stop using it :-). Granted, creating such a translation module is a one-time effort, but it seems inevitable that the current regex module will eventually stop working. If you think keeping regex alive is easier, perhaps you can take it up as a 3rd party module? I really don't want to have to have three different regular expression implementations in the Python distribution forever. (I do understand that deprecating and removing a very heavily used package like regex needs to be done more carefully than e.g. deprecating dircmp, which nobody uses.) > It is acceptable to tell sys-admins of our customers > to install the one extra package when they update Python, > it is not acceptable to require them to change existing code > and it's a maintenance nightmare to provide them with > updated software since it is not actively being maintained > anymore or was specifically written for the customer. Is it acceptable to tell customers not to upgrade to newer Python versions, or to coordinate such upgrades with additional installs from you? [Guido] > > Suppose we decide to deprecate whrandom. The first opportunity to do > > so will be in 2.3. (I think we shouldn't deprecate modules in micro > > releases; the PEP is silent about that but I think it would be a bad > > idea.) Then it will have to continue to exist for at least a year. > > Because whrandom is still in active use (see above), I think the one > > year deprecation period is too short in this case and we should use > > two years. Maybe the warning should only be introduced in the second > > year? That probably means (assuming releases every 6 months): > > > > 2.3 - deprecate in docs > > > > 2.5 - issue warning > > > > 2.7 - move to Lib/lib-old [MAL] > Perhaps we could make Lib/lib-old a separate package ?! I'm not sure what you're proposing. A separate distribution? Or a Python package, which would have to be renamed to e.g. lib_old and from which you could then import it using "from lib_old import whrandom" ? > > Use case #1 is what we usually think of. Some programmer is > > maintaining some Python code. When he upgrades the Python version > > he is using, being the programmer he probably wants to find out > > about deprecated features in the new Python version, so that he > > can fix his code so that it won't break outright in the future. > > IOW, this programmer is glad that there are deprecation warnings > > (compared to the alternative, where he had no idea that he was > > using a deprecated feature until it was removed and his program > > broke outright). > > True; however, deprecation warnings are run-time messages > and not every module in a package is always used by the > programmer, so the warnings may appear after a few months > down the road. Well, that's life. > I would much rather like a tool like PyChecker scan *all* > the code for deprecated modules and maybe a similar tool > for the C API. PyChecker can't check everything. E.g. a source code inspection can't possibly detect int divisions reliably. > > This is annoying: maybe the warning goes to a log file where > > unexpected things cause alarms to go off, maybe it shows up in an > > end user's shell window where it causes confusion, maybe there are > > a lot of warnings and the extra output is simply annoying. These > > users probably aren't Python savvy, so they may panic and call > > customer service. Understandably, nobody is happy. Marc-Andre > > and Fredrik already know their program issues a warning, and > > they've fixed it in their own version, and they don't want to > > waste time telling every user how to disable or ignore the > > warning, or shipping upgrades. > > It not so much about wasting time, it's about bad PR: to the > customer it appears that you've written malfunctioning code. Somewhat granted, though it suggests that the customer doesn't understand what they're talking about. Customers always blame the wrong party (e.g. whenever some Windows game breaks the Python installer, Python gets the blame). > > I would like to suggest that instead, professional software > > distributors like eGenix and PythonWare tweak their distributions to > > turn of warnings by default. This is easy enough: in the main > > program, insert > > > > import warnings > > warnings.filterwarnings("ignore") > > > > at the top (before other imports). If the code needs to run on Python > > 2.0 or before, insert the whole thing inside a try: / except > > ImportError: pass. If you want an easy way to enable warnings, you > > can invent something else (e.g. check an environment variable). > > > > Would this be acceptable? > > Yes. Great! > I would still like the option to continue using code which was > deprecated by PythonLabs. Just because code is unmaintained > doesn't mean that it cannot be used anymore. Granted. > So how about this: > > * deprecated modules get moved into a separate distutils > based package and are made available on python.org together > with the new Python version; the package should contain > a copy of PEP 4 for the curious reader If you want to maintain this, OK. > * deprecated C APIs are grouped in a file pyold.h which > only gets included by Python.h if Py_DEPRECATED is > defined; that header files should contain a list > similar to PEP 4 for the deprecated C APIs and if > possible include a description of how to update existing > code I think this is fine too (though I admit I haven't been following the Py_DEPRECATED discussion in detail). > Both techniques aim for stability: they make the deprecation > process more visible and they provide downgrade paths for > existing installed code. As long as you and others who will benefit from this help out, I think we can do this. --Guido van Rossum (home page: http://www.python.org/~guido/) From Zooko Thu Apr 11 17:44:12 2002 From: Zooko (Zooko) Date: Thu, 11 Apr 2002 09:44:12 -0700 Subject: [Python-Dev] trace.py and the obscurity of Tools/scripts/ (was: Unittest list) In-Reply-To: Message from Skip Montanaro of "Wed, 10 Apr 2002 12:00:56 CDT." <15540.28616.808690.154553@beluga.mojam.com> References: <3CB46566.C9DC62B7@metaslash.com> <15540.28616.808690.154553@beluga.mojam.com> Message-ID: Skip Montanaro wrote: > > My trace.py stuff is actually in the distribution these days at > Tools/scripts/trace.py. It's languished since I first ripped off profile.py > to create it. Actually I rewrote some of it before 2.2. Your mention of it has prompted me to go ahead and submit a patch that has been lurking around in my sandbox for a couple of months, patch 542562 [1]. It is almost unusuably slow. I'm not sure if I'm responsible for that. There is a faster alternative by Gareth Rees [2], but it has other drawbacks and anyway I'm not already familiar with its guts so I don't feel like hacking on it. The final word on that thread was Skip's suggestion of a whole new tech: [3]. The Obscurity of Tools/scripts/ Since even python-dev'ers find ancient copies of trace.py on ftp sites before they find the one in the Tools/scripts/ directory, and since the debian package of Python 2.2 comes without a Tools/scripts/ directory at all, I conclude that the Tools/scripts/ directory it isn't doing its job very well. I suggest it either be killed or fixed. I'm not sure how to do the latter -- link to it from the doc pages? Regards, Zooko [1] http://sourceforge.net/tracker/index.php?func=detail&aid=542562&group_id=5470&atid=305470 [2] http://mail.python.org/pipermail/python-list/2002-January/079730.html [3] http://mail.python.org/pipermail/python-list/2002-January/079763.html Security and Distributed Systems Engineering From martin@v.loewis.de Thu Apr 11 17:32:18 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 11 Apr 2002 18:32:18 +0200 Subject: [Python-Dev] Java-ing Python In-Reply-To: <20020411094405.A409@eye.neb> References: <20020411094405.A409@eye.neb> Message-ID: vmilitaru@sympatico.ca writes: > I would like some feedback on this from the list, and would also like to know > if Python provides some easier way to add a Grammar to the language, without > the need to hack the C source ('cause, incidently, I kinda suck at C). That is not the right place to ask this question: you are asking on how to develop *with* Python, instead of the development *of* Python - use python-list, please. Regards, Martin From aleax@aleax.it Thu Apr 11 17:56:36 2002 From: aleax@aleax.it (Alex Martelli) Date: Thu, 11 Apr 2002 18:56:36 +0200 Subject: [Python-Dev] Can the cgi module be made Unicode-aware? In-Reply-To: References: <15541.6384.351664.886436@12-248-41-177.client.attbi.com> <15541.40385.925752.353594@12-248-41-177.client.attbi.com> Message-ID: On Thursday 11 April 2002 06:26 pm, Martin v. Loewis wrote: ... > Skip Montanaro writes: > > I did some reading before nodding off last night. The tag takes > > an optional "accept-charset" attribute, which can be a list. > > No, it doesn't - that's a proprietary extension. Or, maybe I'm missing > something: where did you find a statement that this is "official" in > any sense? http://www.w3.org/TR/REC-html40/interact/forms.html#h-17.3 > > Adding an "accept-charset" attribute to the does appear to > > have some effect on Content-Type in some instances, but not in all. > > It might depend on the browser, since it's proprietary. It might of course depend on the browser even if it's official. Browsers notoriously comply to "extended subsets" (:-) of w3c recommendations to bewildering extents. Alex From mwh@python.net Thu Apr 11 17:58:06 2002 From: mwh@python.net (Michael Hudson) Date: 11 Apr 2002 17:58:06 +0100 Subject: [Python-Dev] trace.py and the obscurity of Tools/scripts/ (was: Unittest list) In-Reply-To: Zooko's message of "Thu, 11 Apr 2002 09:44:12 -0700" References: <3CB46566.C9DC62B7@metaslash.com> <15540.28616.808690.154553@beluga.mojam.com> Message-ID: <2melhmjhpd.fsf@starship.python.net> Zooko writes: > Since even python-dev'ers find ancient copies of trace.py on ftp > sites before they find the one in the Tools/scripts/ directory, and > since the debian package of Python 2.2 comes without a > Tools/scripts/ directory at all, I conclude that the Tools/scripts/ > directory it isn't doing its job very well. I suggest it either be > killed or fixed. I'm not sure how to do the latter -- link to it > from the doc pages? Kill it and I'll be after you with an axe. Admittedly there's a lot of cruft that should be removed, rewritten or reorganised, but there's some mighty useful stuff there too (eg. logmerge.py). I don't see why I should suffer just because other people don't know where to look... Cheers, M. -- Darned confusing, unless you have that magic ingredient coffee, of which I can pay you Tuesday for a couple pounds of extra-special grind today. -- John Mitchell, 11 Jan 1999 From walter@livinglogic.de Thu Apr 11 18:08:53 2002 From: walter@livinglogic.de (=?UTF-8?B?V2FsdGVyIETDtnJ3YWxk?=) Date: Thu, 11 Apr 2002 19:08:53 +0200 Subject: [Fwd: Re: [Python-Dev] Can the cgi module be made Unicode-aware?] Message-ID: <3CB5C325.3040409@livinglogic.de> Martin v. Loewis wrote: > Skip Montanaro writes: > > >>I did some reading before nodding off last night. The tag takes an >>optional "accept-charset" attribute, which can be a list. > > > No, it doesn't - that's a proprietary extension. Or, maybe I'm missing > something: where did you find a statement that this is "official" in > any sense? http://www.w3.org/TR/html4/interact/forms.html#h-17.3 states that FORM has such an attribute, which specifies the "list of supported charsets". > [...] Bye, Walter Dörwald From martin@v.loewis.de Thu Apr 11 18:14:40 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 11 Apr 2002 19:14:40 +0200 Subject: [Python-Dev] Can the cgi module be made Unicode-aware? In-Reply-To: <200204111702.g3BH2T201692@mira.informatik.hu-berlin.de> References: <15541.6384.351664.886436@12-248-41-177.client.attbi.com> <15541.40385.925752.353594@12-248-41-177.client.attbi.com> <200204111702.g3BH2T201692@mira.informatik.hu-berlin.de> Message-ID: Alex Martelli writes: > http://www.w3.org/TR/REC-html40/interact/forms.html#h-17.3 Thanks for pointing that out. The same document (at #submit-format) also explains that application/x-www-form-urlencoded only supports ASCII, so Skip shouldn't be too surprised that his form fails for non-ASCII text. Regards, Martin From guido@python.org Thu Apr 11 18:28:47 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 11 Apr 2002 13:28:47 -0400 Subject: [Python-Dev] trace.py and the obscurity of Tools/scripts/ (was: Unittest list) In-Reply-To: Your message of "11 Apr 2002 17:58:06 BST." <2melhmjhpd.fsf@starship.python.net> References: <3CB46566.C9DC62B7@metaslash.com> <15540.28616.808690.154553@beluga.mojam.com> <2melhmjhpd.fsf@starship.python.net> Message-ID: <200204111728.g3BHSlB28221@odiug.zope.com> [On Tools/scripts] > Kill it and I'll be after you with an axe. Admittedly there's a lot > of cruft that should be removed, rewritten or reorganised, but there's > some mighty useful stuff there too (eg. logmerge.py). I don't see why > I should suffer just because other people don't know where to look... Maybe more tools should follow the evolution of ndiff and migrate to the library (where ndiff lives on as difflib). --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Thu Apr 11 18:37:54 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 11 Apr 2002 19:37:54 +0200 Subject: [Python-Dev] Deprecating whrandom; and deprecating in general References: <3CB3093C.B7A22727@metaslash.com> <20020409210943.GB16531@panix.com> <3CB3F79D.A143D1C2@lemburg.com> <20020410142824.GC10792@panix.com> <3CB45EE3.E01DD8CE@lemburg.com> <15540.26640.454643.389710@beluga.mojam.com> <3CB47AEB.65B12B3E@lemburg.com> <20020410194920.GA18150@panix.com> <200204102200.g3AM0iQ23326@pcp742651pcs.reston01.va.comcast.net> <3CB5B312.47148FEC@lemburg.com> <200204111643.g3BGhqN25089@odiug.zope.com> Message-ID: <3CB5C9F2.396EAC81@lemburg.com> Guido van Rossum wrote: > > [MAL] > > Some existing scripts still use e.g. regex which is deprecated > > and probably will get removed from the core distribution soon. > > We can't update that code to use re because the script will have > > to expose the exact same semantics as regex provides and I don't > > know how to do this reliably with re (pcre or SRE). Note that > > these scripts are usually tools which again are used by other > > applications, so changing semantics is not an option either. > > I'm not sure I understand. Are these other packages generating > regular expressions in the regex syntax that your script then applies > somewhere? This is indeed a problem -- unless you can keep your > customers on Python <= 2.2 forever, eventually the regex module will > die of old age. No, the problem is that the tools get used in e.g. shell scripts all over the place, so other software relies on the Python script using the regex RE syntax. > Perhaps what's necessary is a translation module that translates the > regex syntax into re syntax? Such a module could be provided as part > of future releases as regex, without deprecation (at least until you > stop using it :-). > > Granted, creating such a translation module is a one-time effort, but > it seems inevitable that the current regex module will eventually stop > working. > > If you think keeping regex alive is easier, perhaps you can take it up > as a 3rd party module? I really don't want to have to have three > different regular expression implementations in the Python > distribution forever. (I do understand that deprecating and removing > a very heavily used package like regex needs to be done more carefully > than e.g. deprecating dircmp, which nobody uses.) I can understand that, but I also think that working software need not be removed altogether. Rather than moving it to some lib-old directory it would be better to add it to a separate distutils package which people who wish to continue to use these modules can install along with the standard Python installation. To make people aware of this package, python.org should link to it. The same could be done for e.g. the Asian codecs. I think it's worthwhile shipping Python as a core package and a standard set of distutils based add-ons. > > It is acceptable to tell sys-admins of our customers > > to install the one extra package when they update Python, > > it is not acceptable to require them to change existing code > > and it's a maintenance nightmare to provide them with > > updated software since it is not actively being maintained > > anymore or was specifically written for the customer. > > Is it acceptable to tell customers not to upgrade to newer Python > versions, or to coordinate such upgrades with additional installs from > you? It's acceptable to tell them that they should install a special add-on package along with Python ... they'll have to do that anyway with the mx extensions, so adding another package to the list wouldn't hurt. > [Guido] > > > Suppose we decide to deprecate whrandom. The first opportunity to do > > > so will be in 2.3. (I think we shouldn't deprecate modules in micro > > > releases; the PEP is silent about that but I think it would be a bad > > > idea.) Then it will have to continue to exist for at least a year. > > > Because whrandom is still in active use (see above), I think the one > > > year deprecation period is too short in this case and we should use > > > two years. Maybe the warning should only be introduced in the second > > > year? That probably means (assuming releases every 6 months): > > > > > > 2.3 - deprecate in docs > > > > > > 2.5 - issue warning > > > > > > 2.7 - move to Lib/lib-old > > [MAL] > > Perhaps we could make Lib/lib-old a separate package ?! > > I'm not sure what you're proposing. A separate distribution? Or a > Python package, which would have to be renamed to e.g. lib_old and > from which you could then import it using "from lib_old import > whrandom" ? Sorry for the confusion. I meant a distutils package, not a Python package. The idea is to move code from lib-old into that a separately downloadable distutils package, e.g. python-libold. People would then first install Python and then add python-libold and possibly python-asiancodecs as needed. distutils makes this easy. > > > Use case #1 is what we usually think of. Some programmer is > > > maintaining some Python code. When he upgrades the Python version > > > he is using, being the programmer he probably wants to find out > > > about deprecated features in the new Python version, so that he > > > can fix his code so that it won't break outright in the future. > > > IOW, this programmer is glad that there are deprecation warnings > > > (compared to the alternative, where he had no idea that he was > > > using a deprecated feature until it was removed and his program > > > broke outright). > > > > True; however, deprecation warnings are run-time messages > > and not every module in a package is always used by the > > programmer, so the warnings may appear after a few months > > down the road. > > Well, that's life. It's specific to the dynamic nature of Python. E.g. in C you'd get linker errors at link time or compiler errors for changed interfaces. > > I would much rather like a tool like PyChecker scan *all* > > the code for deprecated modules and maybe a similar tool > > for the C API. > > PyChecker can't check everything. E.g. a source code inspection can't > possibly detect int divisions reliably. No, but it certainly can check for 'import whrandom'. > > > This is annoying: maybe the warning goes to a log file where > > > unexpected things cause alarms to go off, maybe it shows up in an > > > end user's shell window where it causes confusion, maybe there are > > > a lot of warnings and the extra output is simply annoying. These > > > users probably aren't Python savvy, so they may panic and call > > > customer service. Understandably, nobody is happy. Marc-Andre > > > and Fredrik already know their program issues a warning, and > > > they've fixed it in their own version, and they don't want to > > > waste time telling every user how to disable or ignore the > > > warning, or shipping upgrades. > > > > It not so much about wasting time, it's about bad PR: to the > > customer it appears that you've written malfunctioning code. > > Somewhat granted, though it suggests that the customer doesn't > understand what they're talking about. Customers always blame the > wrong party (e.g. whenever some Windows game breaks the Python > installer, Python gets the blame). True :-/ (this is actually more about politics than understanding of the issues) > > > I would like to suggest that instead, professional software > > > distributors like eGenix and PythonWare tweak their distributions to > > > turn of warnings by default. This is easy enough: in the main > > > program, insert > > > > > > import warnings > > > warnings.filterwarnings("ignore") > > > > > > at the top (before other imports). If the code needs to run on Python > > > 2.0 or before, insert the whole thing inside a try: / except > > > ImportError: pass. If you want an easy way to enable warnings, you > > > can invent something else (e.g. check an environment variable). > > > > > > Would this be acceptable? > > > > Yes. > > Great! > > > I would still like the option to continue using code which was > > deprecated by PythonLabs. Just because code is unmaintained > > doesn't mean that it cannot be used anymore. > > Granted. > > > So how about this: > > > > * deprecated modules get moved into a separate distutils > > based package and are made available on python.org together > > with the new Python version; the package should contain > > a copy of PEP 4 for the curious reader > > If you want to maintain this, OK. I'd need help from other developers. > > * deprecated C APIs are grouped in a file pyold.h which > > only gets included by Python.h if Py_DEPRECATED is > > defined; that header files should contain a list > > similar to PEP 4 for the deprecated C APIs and if > > possible include a description of how to update existing > > code > > I think this is fine too (though I admit I haven't been following the > Py_DEPRECATED discussion in detail). Good. > > Both techniques aim for stability: they make the deprecation > > process more visible and they provide downgrade paths for > > existing installed code. > > As long as you and others who will benefit from this help out, I think > we can do this. Great. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From barry@zope.com Thu Apr 11 18:42:36 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 11 Apr 2002 13:42:36 -0400 Subject: [Python-Dev] trace.py and the obscurity of Tools/scripts/ (was: Unittest list) References: <3CB46566.C9DC62B7@metaslash.com> <15540.28616.808690.154553@beluga.mojam.com> <2melhmjhpd.fsf@starship.python.net> <200204111728.g3BHSlB28221@odiug.zope.com> Message-ID: <15541.51980.403233.710018@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> Maybe more tools should follow the evolution of ndiff and GvR> migrate to the library (where ndiff lives on as difflib). Probably so. I wonder if it makes sense to put them in a package (e.g. a `scripts' package)? -Barry From guido@python.org Thu Apr 11 18:44:19 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 11 Apr 2002 13:44:19 -0400 Subject: [Python-Dev] trace.py and the obscurity of Tools/scripts/ (was: Unittest list) In-Reply-To: Your message of "Thu, 11 Apr 2002 13:42:36 EDT." <15541.51980.403233.710018@anthem.wooz.org> References: <3CB46566.C9DC62B7@metaslash.com> <15540.28616.808690.154553@beluga.mojam.com> <2melhmjhpd.fsf@starship.python.net> <200204111728.g3BHSlB28221@odiug.zope.com> <15541.51980.403233.710018@anthem.wooz.org> Message-ID: <200204111744.g3BHiJw30194@odiug.zope.com> > GvR> Maybe more tools should follow the evolution of ndiff and > GvR> migrate to the library (where ndiff lives on as difflib). > > Probably so. I wonder if it makes sense to put them in a package > (e.g. a `scripts' package)? > > -Barry No, the point is that most of them ought to be converted into a usable library module, whose __main__ is just a simple way to invoke it. Many existing library modules already have this: e.g. pdb, profile, mimify, base64, etc. If you're going to keep them *just* as scripts, then they should remain in Tools. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Apr 11 18:50:41 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 11 Apr 2002 13:50:41 -0400 Subject: [Python-Dev] Deprecating whrandom; and deprecating in general In-Reply-To: Your message of "Thu, 11 Apr 2002 19:37:54 +0200." <3CB5C9F2.396EAC81@lemburg.com> References: <3CB3093C.B7A22727@metaslash.com> <20020409210943.GB16531@panix.com> <3CB3F79D.A143D1C2@lemburg.com> <20020410142824.GC10792@panix.com> <3CB45EE3.E01DD8CE@lemburg.com> <15540.26640.454643.389710@beluga.mojam.com> <3CB47AEB.65B12B3E@lemburg.com> <20020410194920.GA18150@panix.com> <200204102200.g3AM0iQ23326@pcp742651pcs.reston01.va.comcast.net> <3CB5B312.47148FEC@lemburg.com> <200204111643.g3BGhqN25089@odiug.zope.com> <3CB5C9F2.396EAC81@lemburg.com> Message-ID: <200204111750.g3BHofO30223@odiug.zope.com> > > [MAL] > > > Some existing scripts still use e.g. regex which is deprecated > > > and probably will get removed from the core distribution soon. > > > We can't update that code to use re because the script will have > > > to expose the exact same semantics as regex provides and I don't > > > know how to do this reliably with re (pcre or SRE). Note that > > > these scripts are usually tools which again are used by other > > > applications, so changing semantics is not an option either. > > > > I'm not sure I understand. Are these other packages generating > > regular expressions in the regex syntax that your script then applies > > somewhere? This is indeed a problem -- unless you can keep your > > customers on Python <= 2.2 forever, eventually the regex module will > > die of old age. > > No, the problem is that the tools get used in e.g. shell scripts > all over the place, so other software relies on the Python script > using the regex RE syntax. Can you be more concrete? I still don't think I understand it. > > If you think keeping regex alive is easier, perhaps you can take > > it up as a 3rd party module? I really don't want to have to have > > three different regular expression implementations in the Python > > distribution forever. (I do understand that deprecating and > > removing a very heavily used package like regex needs to be done > > more carefully than e.g. deprecating dircmp, which nobody uses.) > > I can understand that, but I also think that working software > need not be removed altogether. Rather than moving it to > some lib-old directory it would be better to add it to a > separate distutils package which people who wish to > continue to use these modules can install along with the > standard Python installation. But enabling lib-old is a lot easier than downloading and installing a separate package! (All it takes is adding it to $PYTHONPATH or sys.path. And this can be done on a per-user basis.) For Python modules, I have no objection to keeping them around in lib-old forever. For C code, I think we should be relieved of the maintenance burden at some point, and then I agree that your separate download solution works fine. > To make people aware of this package, python.org should > link to it. The same could be done for e.g. the Asian > codecs. > > I think it's worthwhile shipping Python as a core package > and a standard set of distutils based add-ons. I expect that most people would never install the add-ons. > > Is it acceptable to tell customers not to upgrade to newer Python > > versions, or to coordinate such upgrades with additional installs from > > you? > > It's acceptable to tell them that they should install a > special add-on package along with Python ... they'll > have to do that anyway with the mx extensions, so > adding another package to the list wouldn't hurt. Good. > Sorry for the confusion. I meant a distutils package, not a > Python package. > > The idea is to move code from lib-old into that a > separately downloadable distutils package, e.g. > python-libold. > > People would then first install Python and then add > python-libold and possibly python-asiancodecs as > needed. distutils makes this easy. Yeah, but most people wouldn't download and install them, and if at some later point they run some 3rd party code and get an import error, they may not realize the solution. > > PyChecker can't check everything. E.g. a source code inspection can't > > possibly detect int divisions reliably. > > No, but it certainly can check for 'import whrandom'. And then again, most Python users have probably never heard of PyChecker. > > > * deprecated modules get moved into a separate distutils > > > based package and are made available on python.org together > > > with the new Python version; the package should contain > > > a copy of PEP 4 for the curious reader > > > > If you want to maintain this, OK. > > I'd need help from other developers. Maybe you can lobby specific individuals who you think may share your goals. --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Thu Apr 11 18:52:55 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 11 Apr 2002 13:52:55 -0400 Subject: [Python-Dev] trace.py and the obscurity of Tools/scripts/ (was: Unittest list) In-Reply-To: <15541.51980.403233.710018@anthem.wooz.org> References: <3CB46566.C9DC62B7@metaslash.com> <15540.28616.808690.154553@beluga.mojam.com> <2melhmjhpd.fsf@starship.python.net> <200204111728.g3BHSlB28221@odiug.zope.com> <15541.51980.403233.710018@anthem.wooz.org> Message-ID: <15541.52599.46047.56754@grendel.zope.com> Barry A. Warsaw writes: > Probably so. I wonder if it makes sense to put them in a package > (e.g. a `scripts' package)? Not sure about the package, but I like having the bulk of each script be in some importable module. Giving it a main() that takes an argv, using sys.argv if None is passed, and returning the return code for the shell is very attractive. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From tim.one@comcast.net Thu Apr 11 19:00:48 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 11 Apr 2002 14:00:48 -0400 Subject: [Python-Dev] trace.py and the obscurity of Tools/scripts/ (was: Unittest list) In-Reply-To: <15541.51980.403233.710018@anthem.wooz.org> Message-ID: [Guido] > Maybe more tools should follow the evolution of ndiff and > migrate to the library (where ndiff lives on as difflib). [Barry] > Probably so. I wonder if it makes sense to put them in a package > (e.g. a `scripts' package)? I don't see how that would be an improvement over leaving them in the scripts directory (which, btw, doesn't bother me a bit). Migration to the library is likely counterproductive unless the code is rewritten to *be* a library. Like ndiff was extensively refactored by two people (me and David Goodger), over two release cycles, to restructure it as a collection of reusable classes and functions. Even so, a much smaller ndiff.py *still* lives in the scripts directory, because the specific application of these algorithms-- and cmdline interface --it supplies don't make sense in a general library module. From barry@zope.com Thu Apr 11 19:07:29 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 11 Apr 2002 14:07:29 -0400 Subject: [Python-Dev] trace.py and the obscurity of Tools/scripts/ (was: Unittest list) References: <15541.51980.403233.710018@anthem.wooz.org> Message-ID: <15541.53473.786848.71301@anthem.wooz.org> >>>>> "TP" == Tim Peters writes: TP> I don't see how that would be an improvement over leaving them TP> in the scripts directory (which, btw, doesn't bother me a TP> bit). TP> Migration to the library is likely counterproductive unless TP> the code is rewritten to *be* a library. Like ndiff was TP> extensively refactored by two people (me and David Goodger), TP> over two release cycles, to restructure it as a collection of TP> reusable classes and functions. Even so, a much smaller TP> ndiff.py *still* lives in the scripts directory, because the TP> specific application of these algorithms-- and cmdline TP> interface --it supplies don't make sense in a general library TP> module. That's what I think I meant,... yeah! IOW, convert the script to a library module, with a main() and leave a little driver in the Tools/scripts directory. I was just thinking that a scripts package might be a convenient place to dump these little thingies. But maybe not. -Barry From fredrik@pythonware.com Thu Apr 11 19:17:19 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 11 Apr 2002 20:17:19 +0200 Subject: [Python-Dev] Deprecating whrandom; and deprecating in general References: <3CB3093C.B7A22727@metaslash.com> <20020409210943.GB16531@panix.com> <3CB3F79D.A143D1C2@lemburg.com> <20020410142824.GC10792@panix.com> <3CB45EE3.E01DD8CE@lemburg.com> <15540.26640.454643.389710@beluga.mojam.com> <3CB47AEB.65B12B3E@lemburg.com> <20020410194920.GA18150@panix.com> <200204102200.g3AM0iQ23326@pcp742651pcs.reston01.va.comcast.net> <3CB5B312.47148FEC@lemburg.com> <200204111643.g3BGhqN25089@odiug.zope.com> Message-ID: <025601c1e185$20928de0$ced241d5@hagrid> Guido van Rossum wrote: > > We can't update that code to use re because the script will have > > to expose the exact same semantics as regex provides and I don't > > know how to do this reliably with re (pcre or SRE). afaik, regex and re has exactly the same semantics. it's the syntax that differs, in various intricate ways. > I'm not sure I understand. Are these other packages generating > regular expressions in the regex syntax that your script then applies > somewhere? This is indeed a problem -- unless you can keep your > customers on Python <= 2.2 forever, eventually the regex module will > die of old age. maybe the libraries take RE patterns as input? > Perhaps what's necessary is a translation module that translates the > regex syntax into re syntax? >>> import reconvert >>> help(reconvert) Help on module reconvert: NAME reconvert - Convert old ("regex") regular expressions to new syntax ("re"). (I'll comment on the rest of this discussion later, when I've rebooted my brain) From guido@python.org Thu Apr 11 19:19:55 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 11 Apr 2002 14:19:55 -0400 Subject: [Python-Dev] PEP 278 - finished? In-Reply-To: Your message of "Fri, 29 Mar 2002 00:16:23 +0100." References: Message-ID: <200204111819.g3BIJtV02956@odiug.zope.com> I'm proposing to accept PEP 278 and let Jack check it in. There will probably be some small issues with the code (though it passes the test suite on my Linux box), but I expect the only way to tease those out is to check it in first. Jack did an admirable job of answering all questions about the patch, adding doc patches and unit tests, and so on. Unless there's significant uproar about this within 48 hours, I'll approve the PEP. One comment for Jack: I think that the 'newlines' attribute should exist even if universal newlines are not configured; it should always be None in that case. --Guido van Rossum (home page: http://www.python.org/~guido/) From Zooko Thu Apr 11 19:19:42 2002 From: Zooko (Zooko) Date: Thu, 11 Apr 2002 11:19:42 -0700 Subject: [Python-Dev] trace.py and the obscurity of Tools/scripts/ (was: Unittest list) In-Reply-To: Message from barry@zope.com (Barry A. Warsaw) of "Thu, 11 Apr 2002 14:07:29 EDT." <15541.53473.786848.71301@anthem.wooz.org> References: <15541.51980.403233.710018@anthem.wooz.org> <15541.53473.786848.71301@anthem.wooz.org> Message-ID: Barry A. Warsaw wrote: > > That's what I think I meant,... yeah! IOW, convert the script to a > library module, with a main() and leave a little driver in the > Tools/scripts directory. So in terms of `trace.py', it is a widely useful tool and already has a programmatic interface. Being added to the hallowed Python Standard Library would be a major step up in publicity and hence usage. It would require better docs regarding the programmatic usage. Anything else that would be required for trace to make this transition? Regards, Zooko Zooko.Com -- Security and Distributed Systems Engineering From guido@python.org Thu Apr 11 19:31:41 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 11 Apr 2002 14:31:41 -0400 Subject: [Python-Dev] trace.py and the obscurity of Tools/scripts/ (was: Unittest list) In-Reply-To: Your message of "Thu, 11 Apr 2002 11:19:42 PDT." References: <15541.51980.403233.710018@anthem.wooz.org> <15541.53473.786848.71301@anthem.wooz.org> Message-ID: <200204111831.g3BIVfn03022@odiug.zope.com> > So in terms of `trace.py', it is a widely useful tool and already > has a programmatic interface. Being added to the hallowed Python > Standard Library would be a major step up in publicity and hence > usage. It would require better docs regarding the programmatic > usage. > > Anything else that would be required for trace to make this transition? I see a copyright notice in the comments; I'd appreciate a transfer to the PSF so we can remove the copyrights. (The CWI copyright can be removed.) It could use a style upgrade. Change log comments typically don't go into comments (we have CVS for that). There are many lines longer than 80 characters (the preferred limit is 72 or 78). It uses strange @param markup in docstrings. There are lots of signed inline comments that sounds more like CVS checkin narrative than help on understanding the source. Other than that, I don't see why it couldn't be given more prominence. --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Thu Apr 11 19:37:00 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 11 Apr 2002 14:37:00 -0400 Subject: [Python-Dev] PEP 278 - finished? References: <200204111819.g3BIJtV02956@odiug.zope.com> Message-ID: <15541.55244.177563.307052@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> One comment for Jack: I think that the 'newlines' attribute GvR> should exist even if universal newlines are not configured; GvR> it should always be None in that case. Minor suggestions: - when mixed newlines are found in a file, can the newlines attribute be a list of those that are found, instead of "mixed"? - shouldn't open() mode "wU" also be illegal? Other than that, +1 (it would squelch the complaints about the email package on systems who's MTA doesn't convert to native-newlines for mail program stdin). -Barry From guido@python.org Thu Apr 11 19:47:03 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 11 Apr 2002 14:47:03 -0400 Subject: [Python-Dev] PEP 278 - finished? In-Reply-To: Your message of "Thu, 11 Apr 2002 14:37:00 EDT." <15541.55244.177563.307052@anthem.wooz.org> References: <200204111819.g3BIJtV02956@odiug.zope.com> <15541.55244.177563.307052@anthem.wooz.org> Message-ID: <200204111847.g3BIl3w03240@odiug.zope.com> > - when mixed newlines are found in a file, can the newlines attribute > be a list of those that are found, instead of "mixed"? Good idea. > - shouldn't open() mode "wU" also be illegal? Indeed. > Other than that, +1 (it would squelch the complaints about the email > package on systems who's MTA doesn't convert to native-newlines for > mail program stdin). Indeed! --Guido van Rossum (home page: http://www.python.org/~guido/) From niemeyer@conectiva.com Thu Apr 11 19:58:26 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Thu, 11 Apr 2002 15:58:26 -0300 Subject: [Python-Dev] PEP 278 - finished? In-Reply-To: <15541.55244.177563.307052@anthem.wooz.org> References: <200204111819.g3BIJtV02956@odiug.zope.com> <15541.55244.177563.307052@anthem.wooz.org> Message-ID: <20020411155826.A8700@ibook.distro.conectiva> > Minor suggestions: > > - when mixed newlines are found in a file, can the newlines attribute > be a list of those that are found, instead of "mixed"? > > - shouldn't open() mode "wU" also be illegal? One more: - Why isn't "rU+" allowed? It's understandable that "+" is not allowed for output, but I can't see a good reason why it shouldn't be allowed to open file for read/write, and read with universal support (maybe some implementation detail?). Allowing this would even easy the task, since when writting to the file the programmer could consider the newlines attribute, if he wants. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From tim.one@comcast.net Thu Apr 11 19:59:35 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 11 Apr 2002 14:59:35 -0400 Subject: [Python-Dev] trace.py and the obscurity of Tools/scripts/ (was: Unittest list) In-Reply-To: Message-ID: [Zooko] > ... > It would require better docs regarding the programmatic usage. I believe it needs much better docs period. I've tried to steer people toward it, but they just can't figure out how to use it. > Anything else that would be required for trace to make this transition? It would help if the bugs were fixed, or identified as common gotchas and explained in the hypothesized better docs . For example, def f(n): sum = 0 for i in range(n): sum for j in range(n): sum += 1 return sum print f(10) produces a .cover file containing 2: def f(n): 2: sum = 0 12: for i in range(n): 10: sum 120: for j in range(n): 100: sum += 1 1: return sum . 1: print f(10) Half the counts reported are "a surprise" to the uninitiated in one way or another; most people seem to expect 1: def f(n): 1: sum = 0 1: for i in range(n): 10: sum 10: for j in range(n): 100: sum += 1 1: return sum . 1: print f(10) while a smaller but still significant number seem to expect 1: def f(n): 1: sum = 0 11: for i in range(n): 10: sum 110: for j in range(n): 100: sum += 1 1: return sum . 1: print f(10) I realize "intuitive" counts are hard to achieve, but if it's in the standard library I'm likely to become the one who has to try to explain "non-intuitive" counts away year after year after year. So, to save myself a couple years of possibly otherwise happy life , I'm -1 unless the docs do a great job of cutting off questions before they get asked. From fredrik@pythonware.com Thu Apr 11 20:10:36 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 11 Apr 2002 21:10:36 +0200 Subject: [Python-Dev] PEP 278 - finished? References: <200204111819.g3BIJtV02956@odiug.zope.com> Message-ID: <02c501c1e18c$9246a960$ced241d5@hagrid> guido wrote: > I'm proposing to accept PEP 278 and let Jack check it in. > > There will probably be some small issues with the code (though it > passes the test suite on my Linux box), but I expect the only way to > tease those out is to check it in first. > > Jack did an admirable job of answering all questions about the patch, > adding doc patches and unit tests, and so on. > > Unless there's significant uproar about this within 48 hours, I'll > approve the PEP. I maintain that a separate constructor (e.g. textfile) would be a much cleaner solution. it's a completely different kind of file object, after all. other than that, I see no problems with concept and implementation (you should have done it this way from the start ;-) From guido@python.org Thu Apr 11 20:07:13 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 11 Apr 2002 15:07:13 -0400 Subject: [Python-Dev] PEP 278 - finished? In-Reply-To: Your message of "Thu, 11 Apr 2002 15:58:26 -0300." <20020411155826.A8700@ibook.distro.conectiva> References: <200204111819.g3BIJtV02956@odiug.zope.com> <15541.55244.177563.307052@anthem.wooz.org> <20020411155826.A8700@ibook.distro.conectiva> Message-ID: <200204111907.g3BJ7D703370@odiug.zope.com> > One more: > > - Why isn't "rU+" allowed? It's understandable that "+" is not allowed > for output, but I can't see a good reason why it shouldn't be allowed > to open file for read/write, and read with universal support (maybe > some implementation detail?). Allowing this would even easy the task, > since when writting to the file the programmer could consider the > newlines attribute, if he wants. This is answered by the PEP: """ A partial output implementation, where strings passed to fp.write() would be converted to use fp.newlines as their line terminated but all other output would not is far too surprising, in my view. Because there is no output support for universal newlines there is also no support for a mode "rU+": the surprise factor of the previous paragraph would hold to an even stronger degree. """ --Guido van Rossum (home page: http://www.python.org/~guido/) From niemeyer@conectiva.com Thu Apr 11 20:18:12 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Thu, 11 Apr 2002 16:18:12 -0300 Subject: [Python-Dev] PEP 278 - finished? In-Reply-To: <200204111907.g3BJ7D703370@odiug.zope.com> References: <200204111819.g3BIJtV02956@odiug.zope.com> <15541.55244.177563.307052@anthem.wooz.org> <20020411155826.A8700@ibook.distro.conectiva> <200204111907.g3BJ7D703370@odiug.zope.com> Message-ID: <20020411161812.A8943@ibook.distro.conectiva> > This is answered by the PEP: > > A partial output implementation, where strings passed to fp.write() > would be converted to use fp.newlines as their line terminated but > all other output would not is far too surprising, in my view. > > Because there is no output support for universal newlines there is > also no support for a mode "rU+": the surprise factor of the > previous paragraph would hold to an even stronger degree. I've read that, but I don't agree with the second paragraph. Universal newline support is available only for input. This sentence is enough to easily predict the behavior in every usage case, including "+" modes. If there's any intent to add support to output, then I'd understand the exclusion, since this would make backwards compatibility possible (users of "rU+" could have their code broken, since they were doing output by hand). Otherwise, it should be allowed, IMO. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From guido@python.org Thu Apr 11 20:29:50 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 11 Apr 2002 15:29:50 -0400 Subject: [Python-Dev] PEP 278 - finished? In-Reply-To: Your message of "Thu, 11 Apr 2002 21:10:36 +0200." <02c501c1e18c$9246a960$ced241d5@hagrid> References: <200204111819.g3BIJtV02956@odiug.zope.com> <02c501c1e18c$9246a960$ced241d5@hagrid> Message-ID: <200204111929.g3BJTom03539@odiug.zope.com> > I maintain that a separate constructor (e.g. textfile) would be > a much cleaner solution. > > it's a completely different kind of file object, after all. Unclear. If files opened in regular text mode and binary mode, in read mode, write mode, append mode, and update mode, are all the same text object, I think Universal newlines are just another minor variation. Also, changing the mode is a morelocalized change (it's all in fileobject.c). > other than that, I see no problems with concept and implementation > (you should have done it this way from the start ;-) Thanks. :) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Apr 11 20:34:55 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 11 Apr 2002 15:34:55 -0400 Subject: [Python-Dev] PEP 278 - finished? In-Reply-To: Your message of "Thu, 11 Apr 2002 16:18:12 -0300." <20020411161812.A8943@ibook.distro.conectiva> References: <200204111819.g3BIJtV02956@odiug.zope.com> <15541.55244.177563.307052@anthem.wooz.org> <20020411155826.A8700@ibook.distro.conectiva> <200204111907.g3BJ7D703370@odiug.zope.com> <20020411161812.A8943@ibook.distro.conectiva> Message-ID: <200204111934.g3BJYtS03552@odiug.zope.com> [PEP 278] > > A partial output implementation, where strings passed to fp.write() > > would be converted to use fp.newlines as their line terminated but > > all other output would not is far too surprising, in my view. > > > > Because there is no output support for universal newlines there is > > also no support for a mode "rU+": the surprise factor of the > > previous paragraph would hold to an even stronger degree. [Gustavo] > I've read that, but I don't agree with the second paragraph. Universal > newline support is available only for input. This sentence is enough to > easily predict the behavior in every usage case, including "+" modes. > > If there's any intent to add support to output, then I'd understand > the exclusion, since this would make backwards compatibility possible > (users of "rU+" could have their code broken, since they were doing > output by hand). Otherwise, it should be allowed, IMO. That's one possible reason not to do this. It could be added later, but I think it's better to disallow it for now. Think YAGNI. --Guido van Rossum (home page: http://www.python.org/~guido/) From martin@v.loewis.de Thu Apr 11 17:28:55 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 11 Apr 2002 18:28:55 +0200 Subject: [Python-Dev] Can the cgi module be made Unicode-aware? In-Reply-To: <15541.39542.449397.709483@anthem.wooz.org> References: <15541.6384.351664.886436@12-248-41-177.client.attbi.com> <15541.39542.449397.709483@anthem.wooz.org> Message-ID: barry@zope.com (Barry A. Warsaw) writes: > If the data is encoded in a non-ascii character set, wouldn't (or > shouldn't) the Content-Type: header include a charset="xxx" parameter? Yes, but none of the browsers does that. The urlencoded form doesn't really support charsets in the first place. The multipart/form-data could, but the browsers don't provide the information (in violation of RFC 1867). Regards, Martin From gmcm@hypernet.com Thu Apr 11 20:44:57 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Thu, 11 Apr 2002 15:44:57 -0400 Subject: [Python-Dev] PEP 278 - finished? In-Reply-To: <20020411161812.A8943@ibook.distro.conectiva> References: <200204111907.g3BJ7D703370@odiug.zope.com> Message-ID: <3CB5AF79.10053.6D01315D@localhost> On 11 Apr 2002 at 16:18, Gustavo Niemeyer wrote: > I've read that, but I don't agree with the second > paragraph. Universal newline support is available > only for input. This sentence is enough to easily > predict the behavior in every usage case, including > "+" modes. How can you reliably update a file if there's no obvious mapping between file position and character position? -- Gordon http://www.mcmillan-inc.com/ From niemeyer@conectiva.com Thu Apr 11 20:56:35 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Thu, 11 Apr 2002 16:56:35 -0300 Subject: [Python-Dev] PEP 278 - finished? In-Reply-To: <3CB5AF79.10053.6D01315D@localhost> References: <200204111907.g3BJ7D703370@odiug.zope.com> <3CB5AF79.10053.6D01315D@localhost> Message-ID: <20020411165635.C9392@ibook.distro.conectiva> Hi Gordon! > How can you reliably update a file if there's no > obvious mapping between file position and > character position? If you're updating a text file, the relation is probably about line position, not file position. AFAICS, this would only be a problem if one wanted to insert something between CR and LF . -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From fredrik@pythonware.com Thu Apr 11 21:01:45 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 11 Apr 2002 22:01:45 +0200 Subject: [Python-Dev] PEP 278 - finished? References: <200204111819.g3BIJtV02956@odiug.zope.com> <02c501c1e18c$9246a960$ced241d5@hagrid> <200204111929.g3BJTom03539@odiug.zope.com> Message-ID: <041601c1e193$ba398b20$ced241d5@hagrid> > Unclear. If files opened in regular text mode and binary mode, in > read mode, write mode, append mode, and update mode, are all the same > text object, I think Universal newlines are just another minor > variation. there's enough "Mode U cannot be combined" stuff in the PEP to make me think that they're slightly more than just a minor variation. I think you can simplify the PEP somewhat by emphasizing the difference. making things simpler is never a bad idea. > Also, changing the mode is a morelocalized change (it's all in > fileobject.c). I'm willing to do the work necessary to provide an alternate factory/constructor. from what I can tell, all that has to be done is is to apply the patch, and do some fairly straight- forward refactoring. (and perhaps some slight API renaming -- the functions in there are clearly too useful to deserve names like Py_UniversalNewlineFgets -- "there is no permanent place in this world for ugly names" ;-) so +1 on checking it in as it stands, but -0 on not touching the result until 2.3 final... From jacobs@penguin.theopalgroup.com Thu Apr 11 21:15:02 2002 From: jacobs@penguin.theopalgroup.com (Kevin Jacobs) Date: Thu, 11 Apr 2002 16:15:02 -0400 (EDT) Subject: [Python-Dev] trace.py and the obscurity of Tools/scripts/ (was: Unittest list) In-Reply-To: Message-ID: On Thu, 11 Apr 2002, Tim Peters wrote: > [Zooko] > > Anything else that would be required for trace to make this transition? > > It would help if the bugs were fixed [...] And there are a ton of problems/bugs in trace.py from the current Python CVS. I've fixed about a dozen so far, and still going. I'll submit diffs as soon as I see the light at the end of the tunnel. Maybe I'm just running trace in a very different way than most and exercising some of the weirder code paths? In spite of the many esteemed contributors and the upcoming fixes from me, the design and the code are very raw and are (IMHO) a long way from being production quality. Of course, if everyone pitches in and we have a code frag-fest, it could conceivably be ready quite soon. Regards, -Kevin -- -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com From guido@python.org Thu Apr 11 21:33:08 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 11 Apr 2002 16:33:08 -0400 Subject: [Python-Dev] PEP 278 - finished? In-Reply-To: Your message of "Thu, 11 Apr 2002 22:01:45 +0200." <041601c1e193$ba398b20$ced241d5@hagrid> References: <200204111819.g3BIJtV02956@odiug.zope.com> <02c501c1e18c$9246a960$ced241d5@hagrid> <200204111929.g3BJTom03539@odiug.zope.com> <041601c1e193$ba398b20$ced241d5@hagrid> Message-ID: <200204112033.g3BKX8o03737@odiug.zope.com> > there's enough "Mode U cannot be combined" stuff in the > PEP to make me think that they're slightly more than just a > minor variation. > > I think you can simplify the PEP somewhat by emphasizing > the difference. making things simpler is never a bad idea. > > > Also, changing the mode is a morelocalized change (it's all in > > fileobject.c). > > I'm willing to do the work necessary to provide an alternate > factory/constructor. from what I can tell, all that has to be > done is is to apply the patch, and do some fairly straight- > forward refactoring. > > (and perhaps some slight API renaming -- the functions in > there are clearly too useful to deserve names like > Py_UniversalNewlineFgets -- "there is no permanent place in this > world for ugly names" ;-) > > so +1 on checking it in as it stands, but -0 on not touching > the result until 2.3 final... OK, good. I'll leave this between you & Jack then. I'll try to make sure you two actually do something about the open items before 2.3 is released! --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Apr 11 21:37:09 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 11 Apr 2002 16:37:09 -0400 Subject: [Python-Dev] trace.py and the obscurity of Tools/scripts/ (was: Unittest list) In-Reply-To: Your message of "Thu, 11 Apr 2002 16:15:02 EDT." References: Message-ID: <200204112037.g3BKb9v03754@odiug.zope.com> [Kevin Jacobs] > And there are a ton of problems/bugs in trace.py from the current Python > CVS. Doesn't surprise me. It's a hairy subject, and judging from the comments embedded in the code, none of the contributors so far have shown a complete grasp of the issues. :-) > I've fixed about a dozen so far, and still going. I'll submit diffs > as soon as I see the light at the end of the tunnel. Maybe I'm just > running trace in a very different way than most and exercising some > of the weirder code paths? Everybody who uses this code uses it in a completely ideosyncratic way. That's what makes it so frustrating to try to unify the changes. > In spite of the many esteemed contributors and the upcoming fixes > from me, the design and the code are very raw and are (IMHO) a long > way from being production quality. Of course, if everyone pitches > in and we have a code frag-fest, it could conceivably be ready quite > soon. Go for it! --Guido van Rossum (home page: http://www.python.org/~guido/) From petrilli@amber.org Thu Apr 11 22:01:46 2002 From: petrilli@amber.org (Christopher Petrilli) Date: Thu, 11 Apr 2002 17:01:46 -0400 Subject: [Python-Dev] Stability Metrics [was Stability and Change] In-Reply-To: <200204111410.g3BEApL25642@pcp742651pcs.reston01.va.comcast.net> Message-ID: On 4/11/02 10:10 AM, "Guido van Rossum" wrote: > I think it's nice to keep track of this for ourselves (despite the > interpretational problems). But I'm strengthening my position about > the use of statistics in communicating to the users: I think that > would be bad. Statistics can easily be misused, misinterpreted, and > countered. One project I used to use a lot of is VTK (Visualization ToolKit). They have a rather useful "quality dashboard" that I thought was a great way to get a grasp of the current state of the development tree: http://public.kitware.com/VTK/Testing/Dashboard/20020411-0300-Nightly/Dashbo ard.html I hope that comes through. Anyway, it runs full regression testing every night and reports all the issues that come up. They actually run it on a cluster of various platforms and showed the stability on each platform as a matrix: http://public.kitware.com/VTK/Testing/Dashboard/20020411-0300-Nightly/TestOv erviewByCount.html Anyway, this I do find useful, if only for a developer who wants to understand issues and where the effects of current development is taking stability. Chris -- | Christopher Petrilli | petrilli@amber.org From Jack.Jansen@oratrix.com Thu Apr 11 22:14:54 2002 From: Jack.Jansen@oratrix.com (Jack Jansen) Date: Thu, 11 Apr 2002 23:14:54 +0200 Subject: [Python-Dev] Threading and callbacks - missing documentation Message-ID: <2B28E3AE-4D91-11D6-936C-003065517236@oratrix.com> Folks, unless I'm looking in completely the wrong place (I looked in "extending and Embedding" and in "Python/C API") there is absolutely no information in the docs on the interaction between callbacks and the GIL. There is preciously little information on callbacks and threading, and it isn't very helpful at that (i.e. there doesn't seem to be a recipy on how to obtain the GIL in a callback routine that is about to call Python code, or how to test whether the current thread holds the GIL already). As I'm currently very interested in this for two separate reasons (one work-related, and another one is that I want to make the Mac toolbox modules release the GIL when appropriate) I could write something up, but only if (a) it isn't documented somewhere already and (b) someone can tell me how it actually works:-). -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From guido@python.org Thu Apr 11 22:26:12 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 11 Apr 2002 17:26:12 -0400 Subject: [Python-Dev] Stability Metrics [was Stability and Change] In-Reply-To: Your message of "Thu, 11 Apr 2002 17:01:46 EDT." References: Message-ID: <200204112126.g3BLQCe04762@odiug.zope.com> > One project I used to use a lot of is VTK (Visualization ToolKit). > They have a rather useful "quality dashboard" that I thought was a > great way to get a grasp of the current state of the development > tree: [...] Great for developers, I agree, but has no value for the current discussion topic (which IMO is something like "how to present an image of Python's stability to more prospective corporate users"). --Guido van Rossum (home page: http://www.python.org/~guido/) From gmcm@hypernet.com Thu Apr 11 23:04:28 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Thu, 11 Apr 2002 18:04:28 -0400 Subject: [Python-Dev] PEP 278 - finished? In-Reply-To: <20020411165635.C9392@ibook.distro.conectiva> References: <3CB5AF79.10053.6D01315D@localhost> Message-ID: <3CB5D02C.29650.6D80EFEF@localhost> On 11 Apr 2002 at 16:56, Gustavo Niemeyer wrote: > > How can you reliably update a file if there's no > > obvious mapping between file position and character > > position? > > If you're updating a text file, the relation is > probably about line position, not file position. That's in the logical file. In the physical file, there may be 1 or 2 character line endings. Mixed. So knowing I want line 7 doesn't help me seek to it on disk. No Windows programmer ever uses "+" without "b" without regret ;-(. -- Gordon http://www.mcmillan-inc.com/ From aahz@pythoncraft.com Fri Apr 12 00:59:27 2002 From: aahz@pythoncraft.com (Aahz) Date: Thu, 11 Apr 2002 19:59:27 -0400 Subject: [Python-Dev] Historical backups? In-Reply-To: <20020411154914.X2471@tummy.com> References: <15536.54772.901785.884218@anthem.wooz.org> <20020407211648.P16962@tummy.com> <15537.4426.451759.73033@anthem.wooz.org> <15537.9537.40053.803853@grendel.zope.com> <20020407233429.S16962@tummy.com> <20020408084706.GY16278@xs4all.nl> <15537.42317.234784.205995@grendel.zope.com> <20020408142251.GB716@xs4all.nl> <15537.48601.753108.808234@anthem.wooz.org> <20020411154914.X2471@tummy.com> Message-ID: <20020411235927.GA27464@panix.com> On Thu, Apr 11, 2002, Sean Reifschneider wrote: > On Mon, Apr 08, 2002 at 11:57:13AM -0400, Barry A. Warsaw wrote: >> >>Feel free to whack anything before the last audited and verified >>backup. Does anybody want to burn some disks before we do? > > Jeeze! Over 7GB... That's going to take me moments on my cable-modem. > ;-P If anyone has some suggestions on what I should rate-limit it to, let > me know and I'll do it... IIRC, those are daily copies. Write a script to pull down just weekly or monthly copies. > Otherwise, I'll burn a few copies of them to CDs, verify them, and send > them around to a few folks. Who wants a copy? Remember that we're talking > physical mail, so I'll need your address -- faxed copies would break my fax > machine. ;-P If nobody else from west coast USA speaks up, I'll take one. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "I disrespectfully agree." --SJM From greg@cosc.canterbury.ac.nz Fri Apr 12 01:05:22 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Fri, 12 Apr 2002 12:05:22 +1200 (NZST) Subject: [Python-Dev] Java-ing Python In-Reply-To: <20020411094405.A409@eye.neb> Message-ID: <200204120005.MAA01091@s454.cosc.canterbury.ac.nz> > what if I replace Python's Grammar file with a Java grammar (for > example, the java grammar included in the ANTLR distribution). Hacked > appropriately to map java source to the tokens defined in python's > Grammar would in theory make python 'understand' java. If you're planning on feeding the resulting parse tree to the existing Python bytecode compiler, you don't stand a chance. The compiler is absolutely dependent on getting a particular parse tree structure, and that structure is derived directly from the grammar. The best you could hope for this way would be to get something which is really Python with a few cosmetic alterations to make it look very vaguely Java-like. There's no way it's going to be able to handle arbitrary real-life Java code. To handle anything like real Java, you'll have to extensively rework the bytecode compiler to match the new grammar. Not to mention deciding what to do about all the semantic differences between Python and Java. If you really want to tackle this, go ahead, it sounds like an interesting project -- but be aware that it will be a LOT of work. Unless you've got an enormous amount of Java to translate, it's going to be faster and easier to just rewrite it by hand! Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Fri Apr 12 01:12:18 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Fri, 12 Apr 2002 12:12:18 +1200 (NZST) Subject: [Python-Dev] Java-ing Python /2 In-Reply-To: <20020411103948.A481@eye.neb> Message-ID: <200204120012.MAA01094@s454.cosc.canterbury.ac.nz> > Still, in my opinion it would be quite an interesting feature if > python could make the addition of other grammars as easily as adding a > new python module. I believe there is an intention to re-design the parsing stuff at some time in the future to make the parse tree more easily accessible and manipulable, so maybe something like this will be possible one day. > A new grammar module would basically be a map of the foreign language > to the python tokens. There's always going to be more to doing such a translation than just token-to-token mappings -- otherwise it would be possible to turn Java into Python with a few simple string substitutions. :-) > And a mechanism to deal with non-mappable features of the foreign > language. Yep, that's the really hard part! Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From aahz@pythoncraft.com Fri Apr 12 01:46:45 2002 From: aahz@pythoncraft.com (Aahz) Date: Thu, 11 Apr 2002 20:46:45 -0400 Subject: [Python-Dev] SF training wheels? Message-ID: <20020412004645.GA4356@panix.com> Some of you may remember that I'm having trouble getting SourceForge to work with Lynx. Part of the reason that I've been hesitant to do much testing is because I don't want to much up the Python database. I've therefore decided to set up a new SF project specifically for the purpose of testing SF. I figure we can use it generically to let people muck about with SF. My question is whether the project should be test-sf or pytest-sf. Thoughts? Did I miss an existing SF project? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "I disrespectfully agree." --SJM From guido@python.org Fri Apr 12 02:25:35 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 11 Apr 2002 21:25:35 -0400 Subject: [Python-Dev] Release Schedules (was Stability & change) In-Reply-To: Your message of "Tue, 09 Apr 2002 01:32:51 EDT." <000c01c1df87$fee0aa00$23b53bd0@othello> References: <000c01c1df87$fee0aa00$23b53bd0@othello> Message-ID: <200204120125.g3C1PZD26384@pcp742651pcs.reston01.va.comcast.net> [Raymond Hettinger] > The anti-change [...] will surely welcome a refactored parser, an > exposed parser, unified ints/longs, and optimized variable access. Whoa, that's exactly where experimental releases would be good. You picked a wonderful set of examples! - Despite all the testing we can do, the new parser will have bugs. (So did the nested scopes code, for several releases, and that was a much less drastic overhaul than writing a whole new parser.) I'm confident that we'll get it right eventually, but there will be some pain for the early adopters; this time not in code breaking for backwards incompatibility reasons, but random breakage because of parser bugs. - The new parser will generate different code in certain corner cases that are not 100% defined by the language spec. This can cause differences in evaluation order, or other differences of interpretation (maybe reject certain silly constructs), and that will break *somebody's* code. - Unified ints/longs is definitely going to break code. repr(2**100) will no longer end in 'L', 1<<31 will return a different value, 0x80000000 will mean something different, and so on. We need to add warnings for these things in 2.3, and only change their meanings in 2.4. But someone who skips a release will miss the warnings. And we can't add the warnings to 2.2.2 or 2.1.4 (too disruptive). - Optimized variable access is definitely going to change semantics in end cases. > IOW, I believe that major, useful changes can be made without > enraging the anti-change crowd. I'm not so sure. :-( > Improving the product, fixing bugs, expanding the library, filling > in missing features, optimizing, and instrumenting aren't the issue. > Just don't mess with the syntax and people won't freak. I wish that were true. Adding bool doesn't change any syntax at all. > Raymond Hettinger > > P.S. The one area I'm less certain about is Deprecation. Phasing > out lambda, map, and filter would please many but may have an > equally strong counter-reaction. It's hard to tell (sometimes I > think I'm the only one who like the functional stuff). Not worth the fight. > P.S.S. I think the intensity of reaction to PEP 285 has to do with > it being central to future programming style. It will affect and > appear in programs across the board. Essentially, this proposal > will be as pervasive as a change to the grammar would be. Maybe. And then again, there are people who believe that no function should return a bool -- so why would they care. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From kamikaze@kuoi.asui.uidaho.edu Fri Apr 12 03:14:55 2002 From: kamikaze@kuoi.asui.uidaho.edu (kamikaze@kuoi.asui.uidaho.edu) Date: 12 Apr 2002 02:14:55 -0000 Subject: [Python-Dev] Re: PEP Politics In-Reply-To: <200204091750.g39Hoot22844@odiug.zope.com> Message-ID: <20020412021455.9528.qmail@kuoi.asui.uidaho.edu> Guido van Rossum spake: >But what am I to do? c.l.py may not be representative, but neither is >python-dev, and I'd like to get feedback from as many folks as >possible. Maybe it's just a matter of wording -- make it clear that >the feedback won't necessarily affect the PEP unless I like what's >being said. But that will still cause certain folks to bitch. Sigh. There are always going to be people who will bitch, because they enjoy bitching. There are always going to be people who will take an instant dislike to any change, because they fear change. There are always going to be people who will automatically reject any PEP you propose, *because* you created Python - I don't understand this behavior at all, but you see it in many fields where the "fans" hate anything new from an artist or writer or inventor. There is nothing you can do to fix them. Don't sweat it, and please keep posting your PEPs where as many people as possible can see and discuss them - most of us have a serious need to know what's coming down the line, and at least some voice in that process. -- Mark Hughes From martin@v.loewis.de Fri Apr 12 08:03:54 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 12 Apr 2002 09:03:54 +0200 Subject: [Python-Dev] Threading and callbacks - missing documentation In-Reply-To: <2B28E3AE-4D91-11D6-936C-003065517236@oratrix.com> References: <2B28E3AE-4D91-11D6-936C-003065517236@oratrix.com> Message-ID: Jack Jansen writes: > As I'm currently very interested in this for two separate reasons (one > work-related, and another one is that I want to make the Mac toolbox > modules release the GIL when appropriate) I could write something up, > but only if (a) it isn't documented somewhere already and (b) someone > can tell me how it actually works:-). The issue in a callback is that you need to obtain a thread state before getting the interpreter lock back. There are two ways to do this: 1. the _tkinter approach: You can guarantee that the call-back occurs in the context of the same thread that originally released the GIL. In that case, save the thread state in a global variable, and restore it inside the callback (see ENTER_TCL/ENTER_PYTHON). 2. the "free threading" approach: You assume that callback may occur in the context of an arbitrary thread, even in a thread that was not originally created by Python. In that case, inside the call-back, use PyThreadState_New to create a fresh thread state, then obtain the GIL for this tread state. When the callback is done, delete the thread state. You need to preserve a pointer to the interpreter somehow in a global variable (or provide other means to locate the interpreter). The advantage of 1 is that tracebacks go right back into the main module, even from the callback; in approach 2, tracebacks go only back to the entry in the callback (the callback appears to come out of nowhere). The advantage of 2 is that it is more general. HTH, Martin From thomas.heller@ion-tof.com Fri Apr 12 08:31:19 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri, 12 Apr 2002 09:31:19 +0200 Subject: [Python-Dev] Threading and callbacks - missing documentation References: <2B28E3AE-4D91-11D6-936C-003065517236@oratrix.com> Message-ID: <042201c1e1f4$09b1ed90$e000a8c0@thomasnotebook> From: "Martin v. Loewis" > > The issue in a callback is that you need to obtain a thread state > before getting the interpreter lock back. There are two ways to do > this: > > 1. the _tkinter approach: You can guarantee that the call-back occurs > in the context of the same thread that originally released the GIL. > In that case, save the thread state in a global variable, and > restore it inside the callback (see ENTER_TCL/ENTER_PYTHON). > > 2. the "free threading" approach: You assume that callback may occur > in the context of an arbitrary thread, even in a thread that was > not originally created by Python. In that case, inside the > call-back, use PyThreadState_New to create a fresh thread state, > then obtain the GIL for this tread state. When the callback is > done, delete the thread state. You need to preserve a pointer to > the interpreter somehow in a global variable (or provide other > means to locate the interpreter). > > The advantage of 1 is that tracebacks go right back into the main > module, even from the callback; in approach 2, tracebacks go only back > to the entry in the callback (the callback appears to come out of > nowhere). The advantage of 2 is that it is more general. > Can't you at least partly combine the advantages of 1 and 2 by using thread local storage for the global variable? I have no idea if there is a portable way to do this... Thomas From martin@v.loewis.de Fri Apr 12 08:08:40 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 12 Apr 2002 09:08:40 +0200 Subject: [Python-Dev] SF training wheels? In-Reply-To: <20020412004645.GA4356@panix.com> References: <20020412004645.GA4356@panix.com> Message-ID: Aahz writes: > Thoughts? Did I miss an existing SF project? For testing SF, I normally use one of the projects that I "own"; a project where not too many people are affected if something breaks. If you don't have such a project yet, I could make you an admin of the ipars project. Regards, Martin From Jack.Jansen@cwi.nl Fri Apr 12 08:58:26 2002 From: Jack.Jansen@cwi.nl (Jack Jansen) Date: Fri, 12 Apr 2002 09:58:26 +0200 Subject: [Python-Dev] PEP 278 - finished? In-Reply-To: <041601c1e193$ba398b20$ced241d5@hagrid> Message-ID: <120BAAD2-4DEB-11D6-84C5-003065517236@cwi.nl> On donderdag, april 11, 2002, at 10:01 , Fredrik Lundh wrote: > so +1 on checking it in as it stands, but -0 on not touching > the result until 2.3 final... I'm +1 on reading the second half of that sentence as a statement that you volunteer to do this:-) (Though I still don't understand why you think a separate constructor is somehow cleaner than a new mode flag, even after reading your rationale. But I have enough other interesting things to do so I don't want to argue this point with you, you can have it if you do the work:-) -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From tim.one@comcast.net Fri Apr 12 09:36:54 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 12 Apr 2002 04:36:54 -0400 Subject: [Python-Dev] Threading and callbacks - missing documentation In-Reply-To: <2B28E3AE-4D91-11D6-936C-003065517236@oratrix.com> Message-ID: [Jack Jansen] > unless I'm looking in completely the wrong place (I looked in > "extending and Embedding" and in "Python/C API") there is > absolutely no information in the docs on the interaction between > callbacks and the GIL. Just the Prime Directive: you must not call back into Python unless you hold the GIL. Terrible things will happen otherwise. Actually getting the GIL can be a nightmare. The worst example in the core is posixmodule.c's _PyPclose, where a thread has to get the GIL, but has no thread state to use, and has no interpreter state to use either (in order to get a thread state). It has to bootstrap all that stuff into existence, then tear it down again when it's done. There are two saving graces in this specific routine that allow the obscure dance to work: (1) it knows for sure that Python has already been initialized; and, (2) it knows for sure that the thread calling it does not hold the GIL at the time of the call. If you can't count on both of those, it can get much harder. Anyway, the comments there will be a real help. > There is preciously little information on callbacks and threading, and > it isn't very helpful at that (i.e. there doesn't seem to be a recipy on > how to obtain the GIL in a callback routine that is about to call Python > code, It would take a small book to cover all the cases that can arise. You can find tens of thousands of words about how to do this in the Thread-SIG archives, some attempts at helper frameworks, and a few forgotten promises to make it easier someday. I believe Mark Hammond has a general set of C++ classes to help with this stuff on Windows, but IIRC they rely on Windows-specific TLS (thread local storage) gimmicks. > or how to test whether the current thread holds the GIL already). Unfortunately, this isn't possible: if you need to do this, you need to build your own conventions for keeping track of who has the GIL. [Thomas Heller, to Martin] > Can't you at least partly combine the advantages of 1 and 2 by using > thread local storage for the global variable? I have no idea if there > is a portable way to do this... There isn't a portable way. The closest Python gets is the little-known PyThreadState_GetDict(), which returns a dict object unique to the thread calling it (so each thread can set up its own key->value mappings for a common set of keys). However, the calling thread must hold the GIL (or, as always, terrible things can happen ). So it's useful only if you already already know whether you have the GIL, and how to get the GIL if you don't have it. From thomas.heller@ion-tof.com Fri Apr 12 10:29:26 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri, 12 Apr 2002 11:29:26 +0200 Subject: [Python-Dev] Threading and callbacks - missing documentation References: Message-ID: <004b01c1e204$89fdc220$e000a8c0@thomasnotebook> From: "Tim Peters" > to make it easier someday. I believe Mark Hammond has a general set of C++ > classes to help with this stuff on Windows, but IIRC they rely on > Windows-specific TLS (thread local storage) gimmicks. Right. Here are the pointers: The class is CEnterLeavePython, near the end in http://www.pythonpros.com/cgi-bin/viewcvs.cgi/PyWin32/win32/src/PyWinTypes.h The helper functions are implemented in http://www.pythonpros.com/cgi-bin/viewcvs.cgi/PyWin32/win32/src/PyWinTypesmodule.cpp Thomas From aahz@pythoncraft.com Fri Apr 12 13:46:15 2002 From: aahz@pythoncraft.com (Aahz) Date: Fri, 12 Apr 2002 08:46:15 -0400 Subject: [Python-Dev] SF training wheels? In-Reply-To: References: <20020412004645.GA4356@panix.com> Message-ID: <20020412124615.GA11186@panix.com> On Fri, Apr 12, 2002, Martin v. Loewis wrote: > Aahz writes: >> >> Thoughts? Did I miss an existing SF project? > > For testing SF, I normally use one of the projects that I "own"; a > project where not too many people are affected if something breaks. If I owned any projects, I'd probably have done that, yeah. > If you don't have such a project yet, I could make you an admin of the > ipars project. That's an option, of course, but I figured it might make sense to have a SF project devoted specifically to learning how to use SF; the only question is whether we want to make it Python-specific. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "I disrespectfully agree." --SJM From Anthony Baxter Fri Apr 12 13:55:05 2002 From: Anthony Baxter (Anthony Baxter) Date: Fri, 12 Apr 2002 22:55:05 +1000 Subject: [Python-Dev] Stability Metrics In-Reply-To: Message from akuchlin@mems-exchange.org of "Thu, 11 Apr 2002 09:04:08 -0400." <20020411090408.B26178@mems-exchange.org> Message-ID: <200204121255.g3CCt5Q26934@mbuna.arbhome.com.au> > The number itself is meaningless as a stability indicator; Skip's > suggestion is that the number should drop with successive releases as > the unfixed bugs get more and more obscure. So 2.2.1 fixes 140 bugs, > and 2.2.2 fixes 70, and 2.2.3 fixes 15, and so on. Counting bugfixes is also meaningless because a high bug count could also indicate either someone's found a new way to identify bugs (e.g a working purify) or else there's been a period of stability which people have used to walk through and fix those old, horrible cases. Anthony From Anthony Baxter Fri Apr 12 13:56:10 2002 From: Anthony Baxter (Anthony Baxter) Date: Fri, 12 Apr 2002 22:56:10 +1000 Subject: [Python-Dev] RELEASED: Python 2.2.1 In-Reply-To: Message from Guido van Rossum of "Thu, 11 Apr 2002 09:18:01 -0400." <200204111318.g3BDI1M25287@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204121256.g3CCuB226952@mbuna.arbhome.com.au> >>> Guido van Rossum wrote > > > Can some who knows how sort out the CVS mess? I can learn, I guess, > > > but I'd rather not risk cocking it up. > > To move a tag, simply retag with the same tag but adding the -F > option. If you retag a whole tree with -F, you only get "T filename" > lines for files where the tag is actually moved. In this case I'd say > leave it. Note well: cvs tag -F works poorly in the presence of deleted files. I prefer to cvs rtag -d (delete) and retag the whole lot. Much safer. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From Anthony Baxter Fri Apr 12 13:57:36 2002 From: Anthony Baxter (Anthony Baxter) Date: Fri, 12 Apr 2002 22:57:36 +1000 Subject: [Python-Dev] PEPs 101 & 102: release procedures In-Reply-To: Message from "Fred L. Drake, Jr." of "Thu, 11 Apr 2002 10:33:20 -0400." <15541.40624.881972.563707@grendel.zope.com> Message-ID: <200204121257.g3CCvaH26989@mbuna.arbhome.com.au> >>> "Fred L. Drake, Jr." wrote > > I've updated PEP 101 and PEP 102 to remove the instructions relating > to the SourceForge file release mechanism. If our contingent of > experienced release managers could review the changes, I'd appreciate > it! Darn - beat me to the checkin. I'll review 102. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From gmcm@hypernet.com Fri Apr 12 14:38:32 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Fri, 12 Apr 2002 09:38:32 -0400 Subject: [Python-Dev] Threading and callbacks - missing documentation In-Reply-To: References: <2B28E3AE-4D91-11D6-936C-003065517236@oratrix.com> Message-ID: <3CB6AB18.28191.70D816B5@localhost> On 12 Apr 2002 at 4:36, Tim Peters wrote: > ... I believe Mark Hammond has a general set of C++ > classes to help with this stuff on Windows, but IIRC > they rely on Windows-specific TLS (thread local > storage) gimmicks. No TLS. Mark's stuff actually relies on being called (at some point) with the GIL so he can grab the interpreter- and thread-states. -- Gordon http://www.mcmillan-inc.com/ From guido@python.org Fri Apr 12 17:42:45 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 12 Apr 2002 12:42:45 -0400 Subject: [Python-Dev] New autoconf in use! In-Reply-To: Your message of "12 Apr 2002 13:07:19 +0200." References: <200204111448.g3BEmQP25970@pcp742651pcs.reston01.va.comcast.net> <200204111545.g3BFj6v24771@odiug.zope.com> Message-ID: <200204121642.g3CGgj010545@pcp742651pcs.reston01.va.comcast.net> I'm moving this thread from python-checkins to -dev, since it's relevant to committers. [Guido] > > As long as it fails cleanly with older versions of autoconf, I'm all > > for using autoconf 2.53 now. [Martin] > Done. At the moment, AC_PREREQ is 2.50, so 2.13 will fail. Even the > changes between 2.52 and 2.53 in the generated configure are huge, so > 2.53 should be the "official" version. > > This could be done in two ways: > - allow people to continue to use, say, 2.52, and just ask that 2.53 > is used when committing; or > - bump AC_PREREQ to 2.53; 2.52 (and earlier) will then refuse to work. > It's your choice. Might as well require 2.53. Installing autoconf isn't exactly hard. > The other issue is caching: autoconf 2.50 will, by default, not > generate a config.cache, since that has caused too many problems. > Again, two options: > - leave it as-is, and suggest that people who want caching use the -C > option of configure (new in 2.5x), or > - hack configure to create a cache file by default (I'm not sure how > to do this) Leave it as it is unless there are many complaints. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Apr 12 18:01:07 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 12 Apr 2002 13:01:07 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library Message-ID: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> A while ago, Greg Ward proposed to add his all-singing-all-dancing option parse, Optik, to the standard library. He's just released version 1.3 and thinks it's ready. According to Greg, the getopt-sig likes Optik. This seems just the kind of thing to add to 2.3 without causing complaints about the rate of language change from the user base. Anyone disagree? --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Fri Apr 12 18:09:24 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 12 Apr 2002 13:09:24 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15543.5316.761699.16705@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> A while ago, Greg Ward proposed to add his GvR> all-singing-all-dancing option parse, Optik, to the standard GvR> library. He's just released version 1.3 and thinks it's GvR> ready. According to Greg, the getopt-sig likes Optik. GvR> This seems just the kind of thing to add to 2.3 without GvR> causing complaints about the rate of language change from the GvR> user base. GvR> Anyone disagree? Is Greg willing to sign the PSF contributor's agreement ? Seriously though, the docs should be converted to .tex and there should probably be some unittests for it. Also, is "optik" the right name for the package in the standard library? It's a great project name, but I'm not so sure that it's as descriptive as it could be (I'm thinking mimelib/email here). "getopt" is taken, but how about "options"? -Barry From martin@v.loewis.de Fri Apr 12 18:10:09 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 12 Apr 2002 19:10:09 +0200 Subject: [Python-Dev] Threading and callbacks - missing documentation In-Reply-To: <042201c1e1f4$09b1ed90$e000a8c0@thomasnotebook> References: <2B28E3AE-4D91-11D6-936C-003065517236@oratrix.com> <042201c1e1f4$09b1ed90$e000a8c0@thomasnotebook> Message-ID: "Thomas Heller" writes: > Can't you at least partly combine the advantages of 1 and 2 by using > thread local storage for the global variable? I have no idea if there > is a portable way to do this... That would work, but, as you already guess: there is no portable way to do this. If you can accept platform-specific code, this is a good idea. Regards, Martin From niemeyer@conectiva.com Fri Apr 12 18:14:40 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Fri, 12 Apr 2002 14:14:40 -0300 Subject: [Python-Dev] New Python developers In-Reply-To: <200204121655.g3CGtHU11160@pcp742651pcs.reston01.va.comcast.net> References: <20020411165141.B9392@ibook.distro.conectiva> <200204112014.g3BKEZc03674@odiug.zope.com> <20020411172602.A9854@ibook.distro.conectiva> <200204112039.g3BKd3C03765@odiug.zope.com> <20020411175557.A10132@ibook.distro.conectiva> <200204112120.g3BLK0f04621@odiug.zope.com> <20020411182735.A11250@ibook.distro.conectiva> <200204120035.g3C0ZLK26188@pcp742651pcs.reston01.va.comcast.net> <20020412111245.A1932@ibook.distro.conectiva> <200204121655.g3CGtHU11160@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020412141440.B3705@ibook.distro.conectiva> [discussion about new python developers and how to join the team] [Guido talking about www.python.org/dev] > > I've been watching it since it was born. It's a great place, with > > valuable information indeed. OTOH, it doesn't provide much information > > for the developer who wants to go further, and get a sourceforge > > account, commit access, etc. What are the conventions followed by the > > developers? Is everything discussed before commiting, or most simple > > stuff is commited and then discussed, if somebody thinks it's wrong. > > Bugs on sourceforge are accepted/rejected and then reviewed by you, > > or maybe just until the developer get a "common sense"? > > I suggest that you write to python-dev with those questions, > suggesting that the answers be added to the /dev/ tree. (Or that the > links to the existing answers be made more prominent. :-) Right now!! ;-)) -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From pobrien@orbtech.com Fri Apr 12 18:20:27 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Fri, 12 Apr 2002 12:20:27 -0500 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <15543.5316.761699.16705@anthem.wooz.org> Message-ID: [Barry A. Warsaw] > "getopt" is taken, but how about "options"? How about "cloptions"? (cl for command line) --- Patrick K. O'Brien Orbtech From martin@v.loewis.de Fri Apr 12 18:15:39 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 12 Apr 2002 19:15:39 +0200 Subject: [Python-Dev] Threading and callbacks - missing documentation In-Reply-To: <004b01c1e204$89fdc220$e000a8c0@thomasnotebook> References: <004b01c1e204$89fdc220$e000a8c0@thomasnotebook> Message-ID: "Thomas Heller" writes: > The helper functions are implemented in > http://www.pythonpros.com/cgi-bin/viewcvs.cgi/PyWin32/win32/src/PyWinTypesmodule.cpp The pthreads equivalent of Windows TLS is pthread_getspecific/setspecific, so you could cover pthreads and Win32. Regards, Martin From guido@python.org Fri Apr 12 18:23:57 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 12 Apr 2002 13:23:57 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: Your message of "Fri, 12 Apr 2002 13:09:24 EDT." <15543.5316.761699.16705@anthem.wooz.org> References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <15543.5316.761699.16705@anthem.wooz.org> Message-ID: <200204121723.g3CHNvk11359@pcp742651pcs.reston01.va.comcast.net> > Is Greg willing to sign the PSF contributor's agreement ? He's a PSF member, so I don't see why not. > Seriously though, the docs should be converted to .tex and there > should probably be some unittests for it. Yup. > Also, is "optik" the right name for the package in the standard > library? It's a great project name, but I'm not so sure that it's as > descriptive as it could be (I'm thinking mimelib/email here). > "getopt" is taken, but how about "options"? Yeah, on the other hand, we've got other modules with kewl names. I'm pretty neutral. "options" sounds reasonable (unless you realize that the concept of options is a lot wider than command line options). --Guido van Rossum (home page: http://www.python.org/~guido/) From niemeyer@conectiva.com Fri Apr 12 18:25:15 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Fri, 12 Apr 2002 14:25:15 -0300 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020412142514.C3705@ibook.distro.conectiva> > A while ago, Greg Ward proposed to add his all-singing-all-dancing > option parse, Optik, to the standard library. He's just released > version 1.3 and thinks it's ready. According to Greg, the getopt-sig > likes Optik. > > This seems just the kind of thing to add to 2.3 without causing > complaints about the rate of language change from the user base. +1. This will boost development of command line parsing algorithms. I'm missing something like this for a long time. Btw, is there any intention of including python-bz2? If so, it should be checked out in those platforms I have no access to. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From fdrake@acm.org Fri Apr 12 18:25:55 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 12 Apr 2002 13:25:55 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <15543.5316.761699.16705@anthem.wooz.org> <200204121723.g3CHNvk11359@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15543.6307.759097.257301@grendel.zope.com> Patrick K. O'Brien writes: > How about "cloptions"? (cl for command line) Guido van Rossum writes: > Yeah, on the other hand, we've got other modules with kewl names. I'm > pretty neutral. "options" sounds reasonable (unless you realize that > the concept of options is a lot wider than command line options). The breadth of meaning for the word "options" is why I don't like that name. "optik" doesn't mean a thing to me either. Perhaps "cli" for "command-line interface"? -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From barry@zope.com Fri Apr 12 18:27:00 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 12 Apr 2002 13:27:00 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <15543.5316.761699.16705@anthem.wooz.org> <200204121723.g3CHNvk11359@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15543.6372.921879.848950@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> Yeah, on the other hand, we've got other modules with kewl GvR> names. I'm pretty neutral. And "optik" will probably be findable if you think "command line options, hmm, let's try `options'", so I'm only -0 on keeping optik. GvR> "options" sounds reasonable (unless you realize that the GvR> concept of options is a lot wider than command line options). Yeah, but it's a land grab, so if Greg gets there quickly... -Barry From fredrik@pythonware.com Fri Apr 12 18:31:22 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Fri, 12 Apr 2002 19:31:22 +0200 Subject: [Python-Dev] Adding Optik to 2.3 standard library References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <15543.5316.761699.16705@anthem.wooz.org> Message-ID: <025e01c1e247$e2626860$ced241d5@hagrid> barry wrote: > "getopt" is taken, but how about "options"? how about "getoptions"? (which implies that "getopt" isn't nearly as complete) can optik be used as a drop-in replacement for getopt? From gward@python.net Fri Apr 12 18:32:31 2002 From: gward@python.net (Greg Ward) Date: Fri, 12 Apr 2002 13:32:31 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <15543.5316.761699.16705@anthem.wooz.org> References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <15543.5316.761699.16705@anthem.wooz.org> Message-ID: <20020412173231.GA20493@gerg.ca> [please *do* cc me on this -- I've disabled my python-dev subscription for a while, as I'm going on holiday soon, but I *do* want to see this thread!] On 12 April 2002, Barry A. Warsaw said: > Is Greg willing to sign the PSF contributor's agreement ? In a flash. > Seriously though, the docs should be converted to .tex and there > should probably be some unittests for it. There are tests, but they don't use any testing framework. Might be a good chance to try out PyUnit and see how it stacks up. ;-) > Also, is "optik" the right name for the package in the standard > library? It's a great project name, but I'm not so sure that it's as > descriptive as it could be (I'm thinking mimelib/email here). > "getopt" is taken, but how about "options"? All of the following have occurred to me: * drop the optik package right into the standard lib as-is, cutesy name and all * same as above, but call it options or option_parser or something (I'm *not* keen on cloptions, sorry) * same as either of the above, but merge it into a single module -- the 'optik' package in Optik 1.3 is only 1,100 source lines, of which 579 are code, in 3 modules * cat optik/lib/*.py >> Lib/getopt.py and edit until it works (Guido wasn't keen on this in private email) * turn 'getopt' into a package that makes the existing getopt stuff available as well as Optik's OptionParser and Option -- eg. to use the new interface, you would do from getopt import OptionParser, ... instead of from optik import OptionParser * variation on the previous one that uses subtle and clever import- time tricks to avoid importing Optik's ~1,000 lines of code unless they are actually needed, to reduce the run-time impact on existing code that just uses the current getopt module The advantages of keeping the name "Optik" are: * it has already had some publicity with this name, notably Stephen Figgins' article in onlamp.com * easier for me to convert the docs and test suite The disadvantages are: * it's not really The Way of the Standard Library * means development of Optik is now tied to Python -- I can't maintain a separate thing called "Optik" (I'm not sure if I want to, though -- there's really not much more to do) I'm pretty open to whatever consensus arises. Greg -- Greg Ward - Linux geek gward@python.net http://starship.python.net/~gward/ Hand me a pair of leather pants and a CASIO keyboard -- I'm living for today! From gward@python.net Fri Apr 12 18:35:02 2002 From: gward@python.net (Greg Ward) Date: Fri, 12 Apr 2002 13:35:02 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <025e01c1e247$e2626860$ced241d5@hagrid> References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <15543.5316.761699.16705@anthem.wooz.org> <025e01c1e247$e2626860$ced241d5@hagrid> Message-ID: <20020412173502.GA20558@gerg.ca> On 12 April 2002, Fredrik Lundh said: > how about "getoptions"? (which implies that > "getopt" isn't nearly as complete) If we're going for a generic-sounding, standard-library-style name, I would prefer "option_parser", because that's the main class in Optik that programmers interact with. Eg. from option_parser import OptionParser, make_option > can optik be used as a drop-in replacement for > getopt? No. Never even occurred to me that anyone would want such a thing! Greg -- Greg Ward - geek-at-large gward@python.net http://starship.python.net/~gward/ Sure, I'm paranoid... but am I paranoid ENOUGH? From fdrake@acm.org Fri Apr 12 18:44:01 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 12 Apr 2002 13:44:01 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <20020412142514.C3705@ibook.distro.conectiva> References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <20020412142514.C3705@ibook.distro.conectiva> Message-ID: <15543.7393.229625.771898@grendel.zope.com> Gustavo Niemeyer writes: > Btw, is there any intention of including python-bz2? If so, it should be > checked out in those platforms I have no access to. Is python-bz2 a bzip2 module, similar to the zlib module? I think it would be nice to have the more space-efficient algorithm available, but I really don't know how many people would use it. In the end, most people seem to want to work with ZIP (or JAR) files, and zlib supports that well enough. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From thomas.heller@ion-tof.com Fri Apr 12 18:49:50 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri, 12 Apr 2002 19:49:50 +0200 Subject: [Python-Dev] Adding Optik to 2.3 standard library References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net><20020412142514.C3705@ibook.distro.conectiva> <15543.7393.229625.771898@grendel.zope.com> Message-ID: <02fe01c1e24a$71d0c850$e000a8c0@thomasnotebook> > but I really don't know how many people would use it. In the end, > most people seem to want to work with ZIP (or JAR) files, and zlib > supports that well enough. Hm. I also want to work with tar and tar.gz files. Is there any intend to add a tarfile module to the Python library? Not that I could do it, since I'm a windows only geek ;-) I only want to read/extract them. Thomas From tim.one@comcast.net Fri Apr 12 18:49:49 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 12 Apr 2002 13:49:49 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <15543.6372.921879.848950@anthem.wooz.org> Message-ID: Since "getopt" is already universally understood as the way to spell "ah, that's the command-line option parser module", I'd rather see Optik get exported from getopt, as getopt.XYZ for some value of XYZ. Then the name XYZ doesn't have to convey "command-line option parser" anymore, but just "another one of those". Heck, make getopt itself a package. From guido@python.org Fri Apr 12 18:57:24 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 12 Apr 2002 13:57:24 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: Your message of "Fri, 12 Apr 2002 19:49:50 +0200." <02fe01c1e24a$71d0c850$e000a8c0@thomasnotebook> References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net><20020412142514.C3705@ibook.distro.conectiva> <15543.7393.229625.771898@grendel.zope.com> <02fe01c1e24a$71d0c850$e000a8c0@thomasnotebook> Message-ID: <200204121757.g3CHvOu11530@pcp742651pcs.reston01.va.comcast.net> > Hm. I also want to work with tar and tar.gz files. Is there any > intend to add a tarfile module to the Python library? Dunno, but it's a pretty simple format, so should be easy. Maybe one of the rookie developers would like to tackle this? --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas.heller@ion-tof.com Fri Apr 12 18:58:42 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri, 12 Apr 2002 19:58:42 +0200 Subject: [Python-Dev] Threading and callbacks - missing documentation References: <2B28E3AE-4D91-11D6-936C-003065517236@oratrix.com><042201c1e1f4$09b1ed90$e000a8c0@thomasnotebook> Message-ID: <032501c1e24b$aed1e3a0$e000a8c0@thomasnotebook> From: "Martin v. Loewis" > "Thomas Heller" writes: > > > Can't you at least partly combine the advantages of 1 and 2 by using > > thread local storage for the global variable? I have no idea if there > > is a portable way to do this... > > That would work, but, as you already guess: there is no portable way > to do this. If you can accept platform-specific code, this is a good > idea. > [in another message] > The pthreads equivalent of Windows TLS is > pthread_getspecific/setspecific, so you could cover pthreads and Win32. Normally I don't care too much about portable code, but in this case (storing and retriving the current threadstate between calls to C code and callbacks from C-code into Python code) the most benefits could only be gained if the Python core and hopefully also extension developers cooperated and used this method. Maybe the Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS macros could be changed to store the threadstate in a global TLS area for Windows and pthreads platforms? Thomas From fdrake@acm.org Fri Apr 12 19:01:45 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 12 Apr 2002 14:01:45 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <02fe01c1e24a$71d0c850$e000a8c0@thomasnotebook> References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <20020412142514.C3705@ibook.distro.conectiva> <15543.7393.229625.771898@grendel.zope.com> <02fe01c1e24a$71d0c850$e000a8c0@thomasnotebook> Message-ID: <15543.8457.73252.609503@grendel.zope.com> Thomas Heller writes: > Hm. I also want to work with tar and tar.gz files. Is there any > intend to add a tarfile module to the Python library? Several tar libraries have been written, and there have been several requests for something like this to be more readily available. I found some old links in mail archives, but they were pretty old, and not all worked. There's a TAR writer in Zope: http://cvs.zope.org/Zope2/lib/python/App/tar.py I don't know if a TAR reader is currently available, though. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From niemeyer@conectiva.com Fri Apr 12 19:04:24 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Fri, 12 Apr 2002 15:04:24 -0300 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <15543.7393.229625.771898@grendel.zope.com> References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <20020412142514.C3705@ibook.distro.conectiva> <15543.7393.229625.771898@grendel.zope.com> Message-ID: <20020412150423.A4581@ibook.distro.conectiva> > Is python-bz2 a bzip2 module, similar to the zlib module? I think it > would be nice to have the more space-efficient algorithm available, It's a mix of what zlib and GzipFile provide. It has functions for dealing with direct (de)compression of data (one shot, and serial), and a complete file interface, inheriting from PyFile_Type. > but I really don't know how many people would use it. In the end, > most people seem to want to work with ZIP (or JAR) files, and zlib > supports that well enough. I don't know either. I can't argue with you about zip files usage. I don't use zip by myself for a long time. But I'm a Linux user, and my environment is probably uncommon. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From thomas.heller@ion-tof.com Fri Apr 12 19:06:37 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri, 12 Apr 2002 20:06:37 +0200 Subject: [Python-Dev] Adding Optik to 2.3 standard library References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net><20020412142514.C3705@ibook.distro.conectiva><15543.7393.229625.771898@grendel.zope.com><02fe01c1e24a$71d0c850$e000a8c0@thomasnotebook> <15543.8457.73252.609503@grendel.zope.com> Message-ID: <034c01c1e24c$ca780660$e000a8c0@thomasnotebook> From: "Fred L. Drake, Jr." > > Thomas Heller writes: > > Hm. I also want to work with tar and tar.gz files. Is there any > > intend to add a tarfile module to the Python library? > > Several tar libraries have been written, and there have been several > requests for something like this to be more readily available. I > found some old links in mail archives, but they were pretty old, and > not all worked. > > There's a TAR writer in Zope: > http://cvs.zope.org/Zope2/lib/python/App/tar.py > > I don't know if a TAR reader is currently available, though. > Currently I use one by Lars Gustäbel http://www.gustaebel.de/lars/tarfile. It is pretty new, but according to the docs seems pretty feature complete (but then, what do I know...). Lars even added back a zipfile-compatible interface on my suggestion. There are a handful others also, but I haven't used them. Thomas From niemeyer@conectiva.com Fri Apr 12 19:07:44 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Fri, 12 Apr 2002 15:07:44 -0300 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <02fe01c1e24a$71d0c850$e000a8c0@thomasnotebook> References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net><20020412142514.C3705@ibook.distro.conectiva> <15543.7393.229625.771898@grendel.zope.com> <02fe01c1e24a$71d0c850$e000a8c0@thomasnotebook> Message-ID: <20020412150744.B4581@ibook.distro.conectiva> > Hm. I also want to work with tar and tar.gz files. Is there any > intend to add a tarfile module to the Python library? [...] There are a couple of implementations around. Shouldn't be hard to add some to the standard library. OTOH, I'm not sure if any of them include GNU extensions. I wouldn't like to see a module in the standard library without them. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From barry@zope.com Fri Apr 12 19:09:47 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 12 Apr 2002 14:09:47 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library References: <15543.6372.921879.848950@anthem.wooz.org> Message-ID: <15543.8939.672949.185213@anthem.wooz.org> >>>>> "TP" == Tim Peters writes: TP> Since "getopt" is already universally understood as the way to TP> spell "ah, that's the command-line option parser module", I'd TP> rather see Optik get exported from getopt, as getopt.XYZ for TP> some value of XYZ. Then the name XYZ doesn't have to convey TP> "command-line option parser" anymore, but just "another one of TP> those". Heck, make getopt itself a package. Ah, +1 Make the existing `getopt' thingie a package, and the current getopt module a sub-module of that package. Name it _getopt.py and export it as getopt.getopt through the package's __init__.py. You probably need to also export getopt.error this way too for backwards compatibility (is there anything else people still use?). Next, expose optik.OptionParser as getopt.Parser. Are there any other public names you'd want to expose? This brings up an interesting meta issue, related to migrating these independent projects back into Python's standard library. My experience with the email package suggests something that could make life a little easier: Allow for a setup.py to live inside the directory it's packaging up. In standalone mimelib, I had to have a mostly useless two level directory structure to support distutils, but that structure isn't convenient for the stdlib. If I could move setup.py into Lib/email then I could easily do development inside the stdlib and then just package up a tarball for independent release whenever I felt like it. All the other package metadata, i.e. README, MANIFEST, etc. could easily live in Lib/email too, although setup.py would have to stitch in the tests, which live in a different hierarchy removed from Lib/email. -Barry From fredrik@pythonware.com Fri Apr 12 19:16:10 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Fri, 12 Apr 2002 20:16:10 +0200 Subject: [Python-Dev] Adding Optik to 2.3 standard library References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <15543.5316.761699.16705@anthem.wooz.org> <025e01c1e247$e2626860$ced241d5@hagrid> <20020412173502.GA20558@gerg.ca> Message-ID: <002301c1e24e$23a336f0$ced241d5@hagrid> Greg Ward wrote: > > how about "getoptions"? (which implies that > > "getopt" isn't nearly as complete) > > If we're going for a generic-sounding, standard-library-style name, I > would prefer "option_parser", because that's the main class in Optik > that programmers interact with. Eg. > > from option_parser import OptionParser, make_option in other words, it should be OptionParser.py, according to the style guide. > > can optik be used as a drop-in replacement for > > getopt? > > No. Never even occurred to me that anyone would want > such a thing! if you want to add something to the standard library, it would be nice if we could take something away, right? From tim.one@comcast.net Fri Apr 12 19:21:17 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 12 Apr 2002 14:21:17 -0400 Subject: [Python-Dev] Threading and callbacks - missing documentation In-Reply-To: <3CB6AB18.28191.70D816B5@localhost> Message-ID: [Tim] > ... I believe Mark Hammond has a general set of C++ > classes to help with this stuff on Windows, but IIRC > they rely on Windows-specific TLS (thread local > storage) gimmicks. [Gordon McMillan] > No TLS. Mark's stuff actually relies on being > called (at some point) with the GIL so he can > grab the interpreter- and thread-states. Yes TLS, and all over the place. Look at the code for any of PyWinThreadState_Ensure, PyWinInterpreterLock_Acquire, PyWinInterpreterLock_Release, PyWinThreadState_Free, PyWinThreadState_Clear or DLLMAIN. TLS is used to remember what a thread's PyThreadState is, and to determine whether a thread already has a PyThreadState (it does iff the TLS slot at dwTlsIndex is non-NULL). Simple example: // Asuming we have a valid thread state, acquire the Python lock. void PyWinInterpreterLock_Acquire() { ThreadData *pData = (ThreadData *)TlsGetValue(dwTlsIndex); PyThreadState *thisThreadState = pData->ts; PyEval_AcquireThread(thisThreadState); } From tim.one@comcast.net Fri Apr 12 19:21:17 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 12 Apr 2002 14:21:17 -0400 Subject: [Python-Dev] Threading and callbacks - missing documentation In-Reply-To: <032501c1e24b$aed1e3a0$e000a8c0@thomasnotebook> Message-ID: [Thomas Heller] > ... > Maybe the Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS macros > could be changed to store the threadstate in a global TLS area for > Windows and pthreads platforms? The Thread-SIG knew what needed to be done, but nobody had time to do it. The only difference now is that nobody remembers what needs to be done, and it would consume so much time to reconstruct all the subtleties that-- once again --nobody would be left with enough time to do it . From thomas.heller@ion-tof.com Fri Apr 12 19:30:00 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri, 12 Apr 2002 20:30:00 +0200 Subject: [Python-Dev] Threading and callbacks - missing documentation References: Message-ID: <03bc01c1e250$0e0a4660$e000a8c0@thomasnotebook> From: "Tim Peters" > [Thomas Heller] > > ... > > Maybe the Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS macros > > could be changed to store the threadstate in a global TLS area for > > Windows and pthreads platforms? > > The Thread-SIG knew what needed to be done, but nobody had time to do it. > The only difference now is that nobody remembers what needs to be done, and > it would consume so much time to reconstruct all the subtleties that-- once > again --nobody would be left with enough time to do it . Time aside, I know that I'm not qualified to implement it, but I would surely be qualified to use it . Thomas From gmcm@hypernet.com Fri Apr 12 20:18:38 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Fri, 12 Apr 2002 15:18:38 -0400 Subject: [Python-Dev] Threading and callbacks - missing documentation In-Reply-To: References: <3CB6AB18.28191.70D816B5@localhost> Message-ID: <3CB6FACE.22883.720F7883@localhost> On 12 Apr 2002 at 14:21, Tim Peters wrote: [Tim] >> ... I believe Mark Hammond has a general set of C++ >> classes to help with this stuff on Windows, but >> IIRC they rely on Windows-specific TLS (thread >> local storage) gimmicks. [Gordon McMillan] > > > No TLS. [Tim] > Yes TLS, and all over the place. Oops, yes. I was thinking of what I did in scxx, which rips off Mark, but doesn't use TLS. Heck, at least I remembered where I ripped it off from. -- Gordon http://www.mcmillan-inc.com/ From neal@metaslash.com Fri Apr 12 20:27:40 2002 From: neal@metaslash.com (Neal Norwitz) Date: Fri, 12 Apr 2002 15:27:40 -0400 Subject: random vs whrandom (was RE: [Python-Dev] Unittests) References: <3CB39D8E.C5E837CE@metaslash.com> <200204100223.g3A2Nvp19644@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CB7352C.D7EFC78E@metaslash.com> Guido van Rossum wrote: > > > I added warnings to statcache & xmllib which are deprecated in the docs. > > Any others need to be done? Should statcache be removed from test_sundry? > > Should any modules be moved into lib-old? posixfile? > > A good exercise would be to do a 1-minute review of every module in > the library and decide for yourself if you'd deprecate it if you were > BDFL. I'll gladly review the list. Here's a long list of issues for dealing with deprecation in the standard library. Recommend to move to lib-old: FCNTL deprecated 2001/05/09 - libundoc.tex (under obsolete-modules) gopherlib in pep 4, before oct 2000 - not deprecated in code or doc posixfile in pep 4, before oct 2000 - not deprecated in code, 1.5 in docs regsub deprecated 2000/12/19 - libundoc.tex (under obsolete-modules) stringold not deprecated in code - not documented anywhere tzparse libundoc.tex (under obsolete-modules) TERMIOS deprecated 2001/02/27 - not documented anywhere Python modules I recommend for deprecation: audiodev C modules I recommend for deprecation (not sure what to do to deprecate these, update PEP 4 and comment out of Setup?): dlmodule mpzmodule pcremodule pypcremodule rotormodule stropmodule yuvconvert # SGI specific modules, is there any use for these anymore??? almodule cdmodule clmodule flmodule fmmodule imgfile sgimodule svmodule Should there be a deprecation schedule or estimate for modules we know will become obsolete to give an approximate time? Regex comes to mind. Should User{Dict, List, String} be deprecated since one can derive from {dict, list, str}? Or perhaps scheduled to be deprecated at some point in the future? Should library uses of UserDict (List & String aren't used) be converted to new instances deriving from {}? UserDict is used in: cgi.FormContentDict cookie.Morsel cookie.BaseCookie os._Environ weakref.WeakKeyDictionary weakref.WeakValueDictionary How should deprecated classes/functions/methods be handled? htmllib.HTMLParser.do_nextid() pty.master_open() pty.slave_open() pstats.Stats.ignore() random.Random.randint() rfc822.AddrlistClass string.zfill() How should deprecated parameter usage be handled? (ie, can call with no params, but should always use 1+ params) Cookie.Cookie() Should we (re)move everything that was deprecated in 1.5.2/1.6 or earlier? Or perhaps even 2.0? This info is from the docs: array.read() - 1.5.1 array.write() - 1.5.1 cmpcache - 1.6 cmp - 1.6 operator.sequenceIncludes - 2.0 operator.isCallable - 2.0 posixfile - 1.5 profile.Stats.ignore() - 1.5.1 random.Random.randint() - 2.0 string.atof() - 2.0 string.atoi() - 2.0 string.atol() - 2.0 sys.exc_traceback - 1.5 tempfile.template - 2.0 sys.exit_thread - 1.5.2 Neal From tim.one@comcast.net Fri Apr 12 20:51:47 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 12 Apr 2002 15:51:47 -0400 Subject: random vs whrandom (was RE: [Python-Dev] Unittests) In-Reply-To: <3CB7352C.D7EFC78E@metaslash.com> Message-ID: [Neal Norwitz, working hard on deprecation issues] I hate to make this a death of a thousand cuts, but ... > ... > Should User{Dict, List, String} be deprecated since one can derive > from {dict, list, str}? No, they're still essential. The advice is to derive from a builtin only if all you want to do is *add* new methods and/or attributes. Trying to override existing __getitem__ etc can be very frustrating, because the core implementation often skips the expensive OO dance of looking for overrides of basic methods (e.g., PyList_SET_ITEM(op, i, v) is currently the straight one-liner (((PyListObject *)(op))->ob_item[i] = (v)) and would be enormously slower if it first had to see whether this is a list subtype overriding __setitem__). But the UserXYZ classes are reliably slow . > Or perhaps scheduled to be deprecated at some point in the future? Not in the foreseeable future. > Should library uses of UserDict (List & String aren't used) > be converted to new instances deriving from {}? No, that would break existing code because the core so often ignores the possibility of overrides. > How should deprecated classes/functions/methods be handled? > ... > random.Random.randint() I recommend to rehabilitate (un-deprecate) this one. It's very popular on the Tutor list, and a newbie who thinks dice faces are numbered from 1 thru 6 isn't in dire need of re-education in my eyes . From guido@python.org Fri Apr 12 21:00:02 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 12 Apr 2002 16:00:02 -0400 Subject: random vs whrandom (was RE: [Python-Dev] Unittests) In-Reply-To: Your message of "Fri, 12 Apr 2002 15:51:47 EDT." References: Message-ID: <200204122000.g3CK02Y12102@pcp742651pcs.reston01.va.comcast.net> > > random.Random.randint() > > I recommend to rehabilitate (un-deprecate) this one. It's very popular on > the Tutor list, and a newbie who thinks dice faces are numbered from 1 thru > 6 isn't in dire need of re-education in my eyes . +1 --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Fri Apr 12 21:01:26 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 12 Apr 2002 16:01:26 -0400 Subject: random vs whrandom (was RE: [Python-Dev] Unittests) References: <3CB39D8E.C5E837CE@metaslash.com> <200204100223.g3A2Nvp19644@pcp742651pcs.reston01.va.comcast.net> <3CB7352C.D7EFC78E@metaslash.com> Message-ID: <15543.15638.162086.825429@anthem.wooz.org> > rfc822.AddrlistClass I still use this one in the email package, since it has a convenient interface. -Barry From neal@metaslash.com Fri Apr 12 21:31:51 2002 From: neal@metaslash.com (Neal Norwitz) Date: Fri, 12 Apr 2002 16:31:51 -0400 Subject: random vs whrandom (was RE: [Python-Dev] Unittests) References: <3CB39D8E.C5E837CE@metaslash.com> <200204100223.g3A2Nvp19644@pcp742651pcs.reston01.va.comcast.net> <3CB7352C.D7EFC78E@metaslash.com> <15543.15638.162086.825429@anthem.wooz.org> Message-ID: <3CB74437.5B2521FE@metaslash.com> "Barry A. Warsaw" wrote: > > > rfc822.AddrlistClass > > I still use this one in the email package, since it has a convenient > interface. Guido deprecated it on 16-Jun-98 (1.28). The docstring is: Note: this class interface is deprecated and may be removed in the future. Use rfc822.AddressList instead. Since AddressList derives from AddrlistClass, could you use that instead? Neal From niemeyer@conectiva.com Fri Apr 12 21:41:48 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Fri, 12 Apr 2002 17:41:48 -0300 Subject: random vs whrandom (was RE: [Python-Dev] Unittests) In-Reply-To: <3CB74437.5B2521FE@metaslash.com> References: <3CB39D8E.C5E837CE@metaslash.com> <200204100223.g3A2Nvp19644@pcp742651pcs.reston01.va.comcast.net> <3CB7352C.D7EFC78E@metaslash.com> <15543.15638.162086.825429@anthem.wooz.org> <3CB74437.5B2521FE@metaslash.com> Message-ID: <20020412174148.K4581@ibook.distro.conectiva> > Since AddressList derives from AddrlistClass, could you use that instead? If so, rfc822.parseaddr() should use it as well. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From niemeyer@conectiva.com Fri Apr 12 21:53:04 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Fri, 12 Apr 2002 17:53:04 -0300 Subject: random vs whrandom (was RE: [Python-Dev] Unittests) In-Reply-To: <3CB7352C.D7EFC78E@metaslash.com> References: <3CB39D8E.C5E837CE@metaslash.com> <200204100223.g3A2Nvp19644@pcp742651pcs.reston01.va.comcast.net> <3CB7352C.D7EFC78E@metaslash.com> Message-ID: <20020412175304.L4581@ibook.distro.conectiva> > dlmodule There are useful constants in this module (RTLD_*). Without them, sys.set/getdlopenflags() is useless. If this module is deprecated, these constants should be moved to somewhere else. > cmp - 1.6 What's the alternative for this today? I use it in at least one algorithm. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From barry@zope.com Fri Apr 12 21:54:09 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 12 Apr 2002 16:54:09 -0400 Subject: random vs whrandom (was RE: [Python-Dev] Unittests) References: <3CB39D8E.C5E837CE@metaslash.com> <200204100223.g3A2Nvp19644@pcp742651pcs.reston01.va.comcast.net> <3CB7352C.D7EFC78E@metaslash.com> <15543.15638.162086.825429@anthem.wooz.org> <3CB74437.5B2521FE@metaslash.com> Message-ID: <15543.18801.282062.449736@anthem.wooz.org> >>>>> "NN" == Neal Norwitz writes: NN> Guido deprecated it on 16-Jun-98 (1.28). The docstring is: | Note: this class interface is deprecated and | may be removed in the future. NN> Use rfc822.AddressList instead. NN> Since AddressList derives from AddrlistClass, could you use NN> that instead? Never mind. :) -Barry From barry@zope.com Fri Apr 12 21:55:54 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 12 Apr 2002 16:55:54 -0400 Subject: random vs whrandom (was RE: [Python-Dev] Unittests) References: <3CB39D8E.C5E837CE@metaslash.com> <200204100223.g3A2Nvp19644@pcp742651pcs.reston01.va.comcast.net> <3CB7352C.D7EFC78E@metaslash.com> <15543.15638.162086.825429@anthem.wooz.org> <3CB74437.5B2521FE@metaslash.com> <20020412174148.K4581@ibook.distro.conectiva> Message-ID: <15543.18906.192055.444893@anthem.wooz.org> >>>>> "GN" == Gustavo Niemeyer writes: >> Since AddressList derives from AddrlistClass, could you use >> that instead? GN> If so, rfc822.parseaddr() should use it as well. I just made this change too. -Barry From guido@python.org Fri Apr 12 22:07:12 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 12 Apr 2002 17:07:12 -0400 Subject: random vs whrandom (was RE: [Python-Dev] Unittests) In-Reply-To: Your message of "Fri, 12 Apr 2002 17:53:04 -0300." <20020412175304.L4581@ibook.distro.conectiva> References: <3CB39D8E.C5E837CE@metaslash.com> <200204100223.g3A2Nvp19644@pcp742651pcs.reston01.va.comcast.net> <3CB7352C.D7EFC78E@metaslash.com> <20020412175304.L4581@ibook.distro.conectiva> Message-ID: <200204122107.g3CL7CO12525@pcp742651pcs.reston01.va.comcast.net> > > cmp - 1.6 > > What's the alternative for this today? I use it in at least one > algorithm. This and dircmp and their caching companions have been unified in filecmp. --Guido van Rossum (home page: http://www.python.org/~guido/) From niemeyer@conectiva.com Fri Apr 12 22:21:20 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Fri, 12 Apr 2002 18:21:20 -0300 Subject: random vs whrandom (was RE: [Python-Dev] Unittests) In-Reply-To: <200204122107.g3CL7CO12525@pcp742651pcs.reston01.va.comcast.net> References: <3CB39D8E.C5E837CE@metaslash.com> <200204100223.g3A2Nvp19644@pcp742651pcs.reston01.va.comcast.net> <3CB7352C.D7EFC78E@metaslash.com> <20020412175304.L4581@ibook.distro.conectiva> <200204122107.g3CL7CO12525@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020412182120.C6625@ibook.distro.conectiva> > > > cmp - 1.6 > > > > What's the alternative for this today? I use it in at least one > > algorithm. > > This and dircmp and their caching companions have been unified in > filecmp. Oops.. I thought you were talking about the cmp() builtin. I'm probably wrong. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From niemeyer@conectiva.com Fri Apr 12 22:29:04 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Fri, 12 Apr 2002 18:29:04 -0300 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <200204121757.g3CHvOu11530@pcp742651pcs.reston01.va.comcast.net> References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net><20020412142514.C3705@ibook.distro.conectiva> <15543.7393.229625.771898@grendel.zope.com> <02fe01c1e24a$71d0c850$e000a8c0@thomasnotebook> <200204121757.g3CHvOu11530@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020412182904.D6625@ibook.distro.conectiva> > > Hm. I also want to work with tar and tar.gz files. Is there any > > intend to add a tarfile module to the Python library? > > Dunno, but it's a pretty simple format, so should be easy. Maybe one > of the rookie developers would like to tackle this? I'll check this out, and add GNU extensions, if none of them have it yet. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From Jack.Jansen@oratrix.com Fri Apr 12 22:59:20 2002 From: Jack.Jansen@oratrix.com (Jack Jansen) Date: Fri, 12 Apr 2002 23:59:20 +0200 Subject: [Python-Dev] Threading and callbacks - missing documentation In-Reply-To: Message-ID: <8ADA7D7C-4E60-11D6-8D5A-003065517236@oratrix.com> Well, as PY_BEGIN_ALLOW_THREADS and PY_END_ALLOW_THREADS currently work, should we really try to fix them with TLS? What I would like to see, though, is a way to get at the thread state iff the current platform has thread local storage. My direct problem (the MacPython/MachoPython one, the work problem is different) is that various MacOS API calls such as WaitNextEvent() can cause callbacks to be called (in the current thread). And as these calls block I'd like to bracket them with PY_BAT/PY_EAT. But I know that in both MacPython and MachoPython I have pthreads, and thus TLS, and thus I can count on using it in the callback routine. Moreover, using the TLS only for callback routines allows me to go one further. If other packages that I use (Tk, for instance) call WaitNextEvent while holding the interpreter lock it could cause on of "my" callbacks to be called. But if we have both the TLS and the old mechanism we can detect the situation that the callback is called from the thread holding the interpreter lock and we can compensate for it. -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From skip@pobox.com Fri Apr 12 23:45:00 2002 From: skip@pobox.com (Skip Montanaro) Date: Fri, 12 Apr 2002 17:45:00 -0500 Subject: [Python-Dev] Can the cgi module be made Unicode-aware? In-Reply-To: References: <15541.6384.351664.886436@12-248-41-177.client.attbi.com> <200204111256.g3BCuQk25210@pcp742651pcs.reston01.va.comcast.net> <15541.40385.925752.353594@12-248-41-177.client.attbi.com> Message-ID: <15543.25452.117148.760625@12-248-41-177.client.attbi.com> >> I did some reading before nodding off last night. The tag >> takes an optional "accept-charset" attribute, which can be a list. Martin> No, it doesn't - that's a proprietary extension. Or, maybe I'm Martin> missing something: where did you find a statement that this is Martin> "official" in any sense? w3.org recommendations: http://www.w3.org/TR/REC-html40/interact/forms.html >> Adding an "accept-charset" attribute to the does appear to >> have some effect on Content-Type in some instances, but not in all. Martin> It might depend on the browser, since it's proprietary. I question your assertion that it's a proprietary attribute, simply because I discovered it on w3.org. The only two browsers I tried it with (Mozilla 0.9.4 and Opera 6.0) both respect it, though as I mentioned, Mozilla doesn't decorate the Content-Type header with its value in the form submission request. >> The cgi programmer can't rely on charset information coming from the >> browser and will need a way to tell the cgi module what the charset >> of the incoming data is. I think FieldStorage and MiniFieldStorage >> need optional charset parameters and I think the charset needs to be >> used from the Content-Type header, if present. Martin> Of course, if you also have uploaded files, this cannot work: Martin> the file data never follow the encoding - only the "text" fields Martin> do. Well, yeah, but that's a case of a multipart deal. Each part could (or should? must?) have its own Content-Type header. Skip From skip@pobox.com Fri Apr 12 23:54:47 2002 From: skip@pobox.com (Skip Montanaro) Date: Fri, 12 Apr 2002 17:54:47 -0500 Subject: [Python-Dev] Can the cgi module be made Unicode-aware? In-Reply-To: References: <15541.6384.351664.886436@12-248-41-177.client.attbi.com> <15541.40385.925752.353594@12-248-41-177.client.attbi.com> <200204111702.g3BH2T201692@mira.informatik.hu-berlin.de> Message-ID: <15543.26039.338580.756090@12-248-41-177.client.attbi.com> Alex> http://www.w3.org/TR/REC-html40/interact/forms.html#h-17.3 Martin> The same document (at #submit-format) also explains that Martin> application/x-www-form-urlencoded only supports ASCII, so S= kip Martin> shouldn't be too surprised that his form fails for non-ASCI= I Martin> text. Are you misinterpreting what part has to be ASCII? If I submit a form containing the word lei=DF it appears that the last letter is not encoded as ß before being urlencoded. Instead, the bytes that represent that character in the de= sired encoding are encoded using the usual % notation. For example, if the charset is Latin-1, the encoded string is "lei%DF", not "lei%26%23223%3= B". That may not be the correct way to do it, but the meager empirical evid= ence I was able to gather from Mozilla and Opera suggests that's how it's do= ne. Skip From skip@pobox.com Sat Apr 13 00:16:21 2002 From: skip@pobox.com (Skip Montanaro) Date: Fri, 12 Apr 2002 18:16:21 -0500 Subject: [Python-Dev] New autoconf in use! In-Reply-To: <200204121642.g3CGgj010545@pcp742651pcs.reston01.va.comcast.net> References: <200204111448.g3BEmQP25970@pcp742651pcs.reston01.va.comcast.net> <200204111545.g3BFj6v24771@odiug.zope.com> <200204121642.g3CGgj010545@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15543.27333.576972.493970@12-248-41-177.client.attbi.com> >> The other issue is caching: autoconf 2.50 will, by default, not >> generate a config.cache, since that has caused too many problems. Guido> Leave it as it is unless there are many complaints. Agreed. Most machines are fast enough nowadays that the absence of config.cache isn't a major stumbling block. If they are that sensitive to the performance reduction, they can install autoconf themselves and generate a local version of the configure script. Skip From skip@pobox.com Sat Apr 13 00:26:31 2002 From: skip@pobox.com (Skip Montanaro) Date: Fri, 12 Apr 2002 18:26:31 -0500 Subject: [Python-Dev] trace.py and the obscurity of Tools/scripts/ (was: Unittest list) In-Reply-To: References: <15541.51980.403233.710018@anthem.wooz.org> <15541.53473.786848.71301@anthem.wooz.org> Message-ID: <15543.27943.6250.555793@12-248-41-177.client.attbi.com> zooko> So in terms of `trace.py', it is a widely useful tool and already zooko> has a programmatic interface. Being added to the hallowed Python zooko> Standard Library would be a major step up in publicity and hence zooko> usage. It would require better docs regarding the programmatic zooko> usage. It's speed cries out for a rewrite of some sort. I haven't thought about it, but I wonder if it could be layered on top of hotshot. Skip From jeremy@zope.com Sat Apr 13 00:34:45 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Fri, 12 Apr 2002 19:34:45 -0400 Subject: [Python-Dev] trace.py and the obscurity of Tools/scripts/ (was: Unittest list) In-Reply-To: <15543.27943.6250.555793@12-248-41-177.client.attbi.com> References: <15541.51980.403233.710018@anthem.wooz.org> <15541.53473.786848.71301@anthem.wooz.org> <15543.27943.6250.555793@12-248-41-177.client.attbi.com> Message-ID: <15543.28437.909937.768107@slothrop.zope.com> >>>>> "SM" == Skip Montanaro writes: zooko> So in terms of `trace.py', it is a widely useful tool and zooko> already has a programmatic interface. Being added to the zooko> hallowed Python Standard Library would be a major step up in zooko> publicity and hence usage. It would require better docs zooko> regarding the programmatic usage. SM> It's speed cries out for a rewrite of some sort. I haven't SM> thought about it, but I wonder if it could be layered on top of SM> hotshot. Can any of the handler methods be re-coded in C? The hotshot changes allow you to install a C function as a trace hook, but doesn't the trace function in trace.py do a fair amount of work? Jeremy From tim.one@comcast.net Sat Apr 13 00:40:22 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 12 Apr 2002 19:40:22 -0400 Subject: [Python-Dev] A "new" kind of leak Message-ID: http://www.python.org/sf/543148 reports a prodiguous leak under 2.1 via import inspect def leak(): frame = inspect.currentframe() while 1: leak() This isn't surprising, since leak() assigns the current frame to a local in the same frame, creating a reference cycle, and frame objects weren't added to cyclic gc until 2.2. Question #1: Is adding frame objects to cyclic gc a candidate for 2.1 backporting? Question #1a: Who's gonna pay Neil to spend his free time doing that <0.5 wink>? What is surprising is that it also leaks *memory* in 2.2 and current CVS, although it doesn't leak *objects* anymore, at least not in the sense you're thinking of: gc does find and reclaim all the frames. Yet the process size grows about 50MB/minute on my box anyway. What happens: frames have their own free list. gc is triggered by a net excess of memory (not free list) allocations over deallocations. When enough frames in cycles have been created, that excess triggers, and gc cleans up all of them. However, the memory isn't freed, instead all the frames end up on the frame object free list. As the program goes on, the entire free list is chewed up to create more frames in cycles, and then we're back where we started: it again takes the same excess of new frame allocations to trigger gc again. Now there are twice as many frames in cycles as before (call it 2*F), and gc again cleans them all up, and they all wind up in the frame object free list -- which is now twice as big as before. The program goes on, and creates 2*F frames in cycles out of the free list before allocating any new memory, and it takes an additional F frames' worth of new frame allocations to trigger gc again. Etc. As time goes on, the size of the frameobject free list grows without bound. Question #2: Is this OK? I think not, as this is a leak in reality. Question #3: Can we use pymalloc to manage frame objects instead? Alas, no: frame objects are too big (on my box, over 350 bytes for a minimal frame object, and pymalloc's limit is 256). Question #4: Can we just get rid of the free list? I don't think so -- a frame object is needed on every function invocation, and saving malloc/free overhead is a real win. An effective solution would be to bound the size of the frameobject free list: pick some maximum N. When frame_dealloc sees that there are already N frames on the free list, it frees the frame memory instead of adding it to the free list. Question #5: What's a good value for N? In the absence of pathologies due to cycles, and in the absence of generators, the number of live frame objects is equal to the current call-stack depth. Since we're just trying to bound worst-case quadratic pathologies in cycle cases, the bound can be high -- say, 1000. A non-pathological program creating thousands of simultaneously active short-lived generators would suffer, but I'm not prepared to go to my grave claiming that's not also pathological . Question #6: If we agree it's a good idea to put on a bound in 2.3, is it a good idea to backport the bound? I think yes to 2.2, but no to 2.1. In 2.1 the frame-cycle cases leak anyway. From skip@pobox.com Sat Apr 13 00:45:07 2002 From: skip@pobox.com (Skip Montanaro) Date: Fri, 12 Apr 2002 18:45:07 -0500 Subject: [Python-Dev] trace.py and the obscurity of Tools/scripts/ (was: Unittest list) In-Reply-To: <15543.28437.909937.768107@slothrop.zope.com> References: <15541.51980.403233.710018@anthem.wooz.org> <15541.53473.786848.71301@anthem.wooz.org> <15543.27943.6250.555793@12-248-41-177.client.attbi.com> <15543.28437.909937.768107@slothrop.zope.com> Message-ID: <15543.29059.782535.860209@12-248-41-177.client.attbi.com> SM> It's speed cries out for a rewrite of some sort. I haven't thought SM> about it, but I wonder if it could be layered on top of hotshot. Jeremy> Can any of the handler methods be re-coded in C? The hotshot Jeremy> changes allow you to install a C function as a trace hook, but Jeremy> doesn't the trace function in trace.py do a fair amount of work? Dunno. As you might guess I haven't looked at the trace code in quite awhile... :-) Skip From guido@python.org Sat Apr 13 01:09:46 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 12 Apr 2002 20:09:46 -0400 Subject: [Python-Dev] New autoconf in use! In-Reply-To: Your message of "Fri, 12 Apr 2002 18:16:21 CDT." <15543.27333.576972.493970@12-248-41-177.client.attbi.com> References: <200204111448.g3BEmQP25970@pcp742651pcs.reston01.va.comcast.net> <200204111545.g3BFj6v24771@odiug.zope.com> <200204121642.g3CGgj010545@pcp742651pcs.reston01.va.comcast.net> <15543.27333.576972.493970@12-248-41-177.client.attbi.com> Message-ID: <200204130009.g3D09ku12997@pcp742651pcs.reston01.va.comcast.net> > >> The other issue is caching: autoconf 2.50 will, by default, not > >> generate a config.cache, since that has caused too many problems. > > Guido> Leave it as it is unless there are many complaints. > > Agreed. Most machines are fast enough nowadays that the absence of > config.cache isn't a major stumbling block. If they are that > sensitive to the performance reduction, they can install autoconf > themselves and generate a local version of the configure script. Here's another reason why it's fine to leave the cache off by default: most people will build Python only once per directory -- they'll download a version, run ./configure, make, and make install. For them, the cache doesn't help at all. The people for whom it matters are developers using CVS, who need to run configure frequently (e.g. each time a new configure script is checked in). Those people know what they're doing and can do the -C thing if they find it too slow. Personally, I usually do "./configure; make test" in one window and continue deleting email in another. ;-) --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Sat Apr 13 01:10:33 2002 From: skip@pobox.com (Skip Montanaro) Date: Fri, 12 Apr 2002 19:10:33 -0500 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15543.30585.922471.967802@12-248-41-177.client.attbi.com> Guido> A while ago, Greg Ward proposed to add his Guido> all-singing-all-dancing option parse, Optik, to the standard Guido> library. He's just released version 1.3 and thinks it's ready. Guido> According to Greg, the getopt-sig likes Optik. The only think I remember from the previous discussion was that "Optik" wasn't exactly suggestive of the module's function. I don't recall if any alternatives were suggested at the time though. How about "optionparser" or "optionlib"? Skip From guido@python.org Sat Apr 13 01:21:35 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 12 Apr 2002 20:21:35 -0400 Subject: [Python-Dev] A "new" kind of leak In-Reply-To: Your message of "Fri, 12 Apr 2002 19:40:22 EDT." References: Message-ID: <200204130021.g3D0LZc19139@pcp742651pcs.reston01.va.comcast.net> > http://www.python.org/sf/543148 > > reports a prodiguous leak under 2.1 via > > import inspect > > def leak(): > frame = inspect.currentframe() > > while 1: > leak() You can get the same effect with sys._getframe() instead of inspect.currentframe(). > This isn't surprising, since leak() assigns the current frame to a > local in the same frame, creating a reference cycle, and frame > objects weren't added to cyclic gc until 2.2. Question #1: Is > adding frame objects to cyclic gc a candidate for 2.1 backporting? > Question #1a: Who's gonna pay Neil to spend his free time doing that > <0.5 wink>? If Neil won't, someone else can do it. Adding GC to an object isn't hard, and the tp_traverse/tp_clear implementations probably port right over. [Cute explanation snipped] > Question #2: Is this OK? I think not, as this is a leak in reality. Agreed. > Question #3: Can we use pymalloc to manage frame objects instead? > Alas, no: frame objects are too big (on my box, over 350 bytes for a > minimal frame object, and pymalloc's limit is 256). How would it help? > Question #4: Can we just get rid of the free list? I don't think so > -- a frame object is needed on every function invocation, and saving > malloc/free overhead is a real win. Yup. Try it though. > An effective solution would be to bound the size of the frameobject > free list: pick some maximum N. When frame_dealloc sees that there > are already N frames on the free list, it frees the frame memory > instead of adding it to the free list. Good idea. > Question #5: What's a good value for N? In the absence of > pathologies due to cycles, and in the absence of generators, the > number of live frame objects is equal to the current call-stack > depth. Since we're just trying to bound worst-case quadratic > pathologies in cycle cases, the bound can be high -- say, 1000. A > non-pathological program creating thousands of simultaneously active > short-lived generators would suffer, but I'm not prepared to go to > my grave claiming that's not also pathological . I bet even lowering it to 50 would work just as well in practice. I'm somewhat wary of solutions that use a really high bound for cases like this; the code that deals with exceeding the bound would almost never get executed, so bugs there would linger forever. I think we had this problem with the code for comparing self-referential containers; originally the limit was 1000, now it's something like 20. > Question #6: If we agree it's a good idea to put on a bound in 2.3, > is it a good idea to backport the bound? I think yes to 2.2, but no > to 2.1. In 2.1 the frame-cycle cases leak anyway. But it was reported for 2.1, right? I could imagine making this a showcase of the new religion: more support for older versions. 2.1 is the limit though. --Guido van Rossum (home page: http://www.python.org/~guido/) From nas@python.ca Sat Apr 13 02:24:14 2002 From: nas@python.ca (Neil Schemenauer) Date: Fri, 12 Apr 2002 18:24:14 -0700 Subject: [Python-Dev] A "new" kind of leak In-Reply-To: <200204130021.g3D0LZc19139@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Fri, Apr 12, 2002 at 08:21:35PM -0400 References: <200204130021.g3D0LZc19139@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020412182414.A12127@glacier.arctrix.com> Guido van Rossum wrote: > Adding GC to an object isn't hard, and the tp_traverse/tp_clear > implementations probably port right over. frameobject is not easy. They use a free list, realloc and the trashcan. The 2.1 GC API is a lot harder to use correctly as well. :-( Neil From aahz@pythoncraft.com Sat Apr 13 02:44:43 2002 From: aahz@pythoncraft.com (Aahz) Date: Fri, 12 Apr 2002 21:44:43 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <15543.30585.922471.967802@12-248-41-177.client.attbi.com> References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <15543.30585.922471.967802@12-248-41-177.client.attbi.com> Message-ID: <20020413014443.GA515@panix.com> On Fri, Apr 12, 2002, Skip Montanaro wrote: > > The only think I remember from the previous discussion was that "Optik" > wasn't exactly suggestive of the module's function. I don't recall if any > alternatives were suggested at the time though. > > How about "optionparser" or "optionlib"? How about "argparse"? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ What if there were no rhetorical questions? From barry@zope.com Sat Apr 13 03:05:00 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 12 Apr 2002 22:05:00 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <15543.30585.922471.967802@12-248-41-177.client.attbi.com> Message-ID: <15543.37452.98720.899406@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: SM> How about "optionparser" or "optionlib"? I really think we should expose Optik's main parser class through getopt, which we'd make into a package. I didn't see any response to my previous suggestion, does everybody else hate it? ;) -Barry From Thanks4asking@alliedmarketing.net Sat Apr 13 03:06:30 2002 From: Thanks4asking@alliedmarketing.net (Thanks4asking@alliedmarketing.net) Date: Fri, 12 Apr 2002 22:06:30 -0400 Subject: [Python-Dev] Webmaster Information Message-ID: <4D4508C2-4E5A-11D6-8F4E-00500471CA25@oXhzfV3c> ------=_NextPart_000_00H3_24K45N6P.P2676U87 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: base64 PEhUTUw+DQo8SEVBRD4NCiAgPFRJVExFPlRoYW5rcy00LUFza2luZzwvVElUTEU+DQogIDxMSU5L IHJlbD0ic3R5bGVzaGVldCIgaHJlZj0iaHR0cDovL2NvZ25pZ2VuLm5ldC9jb3Jwb3JhdGUvY29n bmlnZW4uY3NzIj4NCjwvSEVBRD4NCjxCT0RZIEJHQ09MT1I9IiNmZmZmZmYiPg0KPENFTlRFUj4N CiAgPFRBQkxFIENFTExTUEFDSU5HPSIwIiBDRUxMUEFERElORz0iMCIgQUxJR049IkNlbnRlciJ3 aWR0aD01MDA+DQogICAgPFRSPg0KICAgICAgPFREPjxUQUJMRSBDRUxMU1BBQ0lORz0iMCIgQ0VM TFBBRERJTkc9IjAiPg0KCSAgPFRSPg0KCSAgICA8VEQgYmdjb2xvcj0jMDAwMDAwPjxQIEFMSUdO PUNlbnRlcj4NCgkgICAgICA8SU1HIFNSQz0icGljdDQ1LmpwZyIgQUxJR049IkJvdHRvbSIgQUxU PSJbSW1hZ2VdIiBXSURUSD0iMzY1IiBIRUlHSFQ9IjcyIj48L1REPg0KCSAgPC9UUj4NCgkgIDxU Uj4NCgkgICAgPFREPjxUQUJMRSBDRUxMU1BBQ0lORz0iMCIgQ0VMTFBBRERJTkc9IjAiPg0KCQk8 VFI+DQoJCSAgPFREPjxUQUJMRSBDRUxMU1BBQ0lORz0iMCIgQ0VMTFBBRERJTkc9IjAiPg0KCQkg ICAgICA8VFI+DQoJCQk8VEQ+PFRBQkxFIENFTExTUEFDSU5HPSIwIiBDRUxMUEFERElORz0iMCI+ DQoJCQkgICAgPFRSPg0KCQkJICAgICAgPFREPg0KCQkJCSAgPEhSPg0KCQkJICAgICAgPC9URD4N CgkJCSAgICA8L1RSPg0KCQkJICAgIDxUUj4NCgkJCSAgICAgIDxURD48VEFCTEUgQ0VMTFNQQUNJ Tkc9IjAiIENFTExQQURESU5HPSIwIj4NCgkJCQkgIDxUUj4NCgkJCQkgICAgPFREPjxUQUJMRSBD RUxMU1BBQ0lORz0iMCIgQ0VMTFBBRERJTkc9IjAiPg0KCQkJCQk8VFI+DQoJCQkJCSA8VEQ+PEI+ PFNQQU4gY2xhc3M9ImdlbmVyYWwiPjxCSUc+PEJJRz48QklHPkludGVybmV0IFdlYiBUcmFmZmlj IEZvcg0KCQkJCQkgIFNhbGU8L0JJRz48L0JJRz48L0JJRz48L1NQQU4+PC9CPjwvVEQ+DQoJCQkJ CTwvVFI+DQoJCQkJCTxUUj4NCgkJCQkJIDxURD48U1BBTiBjbGFzcz0iZ2VuZXJhbCI+PEk+U3Bl Y2lhbCBQcmljaW5nIEZvciBUcmFmZmljIFRvIFlvdXIgV2ViDQoJCQkJCSAgU2l0ZTo8L0k+PC9T UEFOPjwvVEQ+DQoJCQkJCTwvVFI+DQoJCQkJCTxUUj4NCgkJCQkJIDxURD4mbmJzcDs8L1REPg0K CQkJCQk8L1RSPg0KCQkJCQk8VFI+DQoJCQkJCSA8VEQ+PFNQQU4gY2xhc3M9ImdlbmVyYWwiPjxC Pk1haW5zdHJlYW0gU2l0ZXMgRXhpdCBvciBQb3B1bmRlcg0KCQkJCQkgIFRyYWZmaWM6PC9CPjwv U1BBTj48L1REPg0KCQkJCQk8L1RSPg0KCQkJCQk8VFI+DQoJCQkJCSA8VEQ+PFNQQU4gY2xhc3M9 ImdlbmVyYWwiPjxJPkdlbmVyaWMgVHJhZmZpYyAtICQzLjI1IENQTTwvST48L1NQQU4+PC9URD4N CgkJCQkJPC9UUj4NCgkJCQkJPFRSPg0KCQkJCQkgPFREPjxTUEFOIGNsYXNzPSJnZW5lcmFsIj48 ST5DYXRlZ29yeSBTcGVjaWZpYyBUcmFmZmljIC0gJDUuMDAgQ1BNPC9JPjwvU1BBTj48L1REPg0K CQkJCQk8L1RSPg0KCQkJCQk8VFI+DQoJCQkJCSA8VEQ+Jm5ic3A7PC9URD4NCgkJCQkJPC9UUj4N CgkJCQkJPFRSPg0KCQkJCQkgPFREPjxTUEFOIGNsYXNzPSJnZW5lcmFsIj48Qj5BZHVsdCBTaXRl cyBFeGl0IG9yIFBvcHVuZGVyIFRyYWZmaWM6PC9CPjwvU1BBTj48L1REPg0KCQkJCQk8L1RSPg0K CQkJCQk8VFI+DQoJCQkJCSA8VEQ+PFNQQU4gY2xhc3M9ImdlbmVyYWwiPjxJPkZyb20gUGF5c2l0 ZXMgLSAkNS4wMCBDUE08L0k+PC9TUEFOPjwvVEQ+DQoJCQkJCTwvVFI+DQoJCQkJCTxUUj4NCgkJ CQkJIDxURD48U1BBTiBjbGFzcz0iZ2VuZXJhbCI+PEk+RnJvbSBGcmVlc2l0ZXMgLSAkNC4wMCBD UE08L0k+PC9TUEFOPjwvVEQ+DQoJCQkJCTwvVFI+DQoJCQkJCTxUUj4NCgkJCQkJIDxURD4mbmJz cDs8L1REPg0KCQkJCQk8L1RSPg0KCQkJCQk8VFI+DQoJCQkJCSA8VEQ+PFNQQU4gY2xhc3M9Imdl bmVyYWwiPjxCPlNlYXJjaCBFbmdpbmUgVHJhZmZpYyBBdmFpYWxibGUgVXBvbiBSZXF1ZXN0Lg0K CQkJCQkgIEZvciBBIFF1b3RlIE9uIFNlYXJjaCBFbmdpbmUgVHJhZmZpYyBQbGVhc2UgY2FsbCB1 cyBhdCA5NzMuOTkyLjM5ODUgb3IgZW1haWwNCgkJCQkJICA8QSBIUkVGPSJtYWlsdG86dHJhZmZp Y0BhbGxpZWRtYXJrZXRpbmcubmV0Ij50cmFmZmljQGFsbGllZG1hcmtldGluZy5uZXQ8L0E+PC9C PjwvU1BBTj48L1REPg0KCQkJCQk8L1RSPg0KCQkJCQk8VFI+DQoJCQkJCSA8VEQ+Jm5ic3A7PC9U RD4NCgkJCQkJPC9UUj4NCgkJCQkJPFRSPg0KCQkJCQkgPFREPjxTUEFOIGNsYXNzPSJnZW5lcmFs Ij48Qj5Gb3IgQWRkaXRpb25hbCBJbmZvcm1hdGlvbiBBYm91dCBQdXJjaGFzaW5nDQoJCQkJCSAg VHJhZmZpYyBUaHJvdWdoIEFsbGllZCBJbnRlcm5ldCBNYXJrZXRpbmcgUGxlYXNlIFZpc2l0DQoJ CQkJCSAgPEEgSFJFRj0iaHR0cDovL3RyYWZmaWMuYWxsaWVkbWFya2V0aW5nLm5ldCIgVEFSR0VU PSJfYmxhbmsiPmh0dHA6Ly90cmFmZmljLmFsbGllZG1hcmtldGluZy5uZXQ8L0E+PC9CPjwvU1BB Tj48L1REPg0KCQkJCQk8L1RSPg0KCQkJCQk8VFI+DQoJCQkJCSA8VEQ+DQoJCQkJCSAgIDxIUj4N CgkJCQkJIDwvVEQ+DQoJCQkJCTwvVFI+DQoJCQkJCTxUUj4NCgkJCQkJIDxURD4mbmJzcDs8L1RE Pg0KCQkJCQk8L1RSPg0KCQkJCQk8VFI+DQoJCQkJCSA8VEQ+PEI+PFNQQU4gY2xhc3M9ImdlbmVy YWwiPjxCSUc+PEJJRz48QklHPk9wdC1JbiBFbWFpbCBUcmFmZmljIEZvcg0KCQkJCQkgIFNhbGU8 L0JJRz48L0JJRz48L0JJRz48L1NQQU4+PC9CPjwvVEQ+DQoJCQkJCTwvVFI+DQoJCQkJCTxUUj4N CgkJCQkJIDxURD48U1BBTiBjbGFzcz0iZ2VuZXJhbCI+PEk+U3BlY2lhbCBQcmljaW5nIEZvciBF bWFpbCBUcmFmZmljIFRvIFlvdXIgV2ViDQoJCQkJCSAgU2l0ZTo8L0k+PC9TUEFOPjwvVEQ+DQoJ CQkJCTwvVFI+DQoJCQkJCTxUUj4NCgkJCQkJIDxURD4mbmJzcDsgJm5ic3A7PC9URD4NCgkJCQkJ PC9UUj4NCgkJCQkJPFRSPg0KCQkJCQkgPFREPjxTUEFOIGNsYXNzPSJnZW5lcmFsIj48Qj5XZSBo YXZlIDcgZGlmZmVyZW50IE5ld3NsZXR0ZXJzIHRvIHNlbGVjdA0KCQkJCQkgIGZyb206PC9CPjwv U1BBTj48L1REPg0KCQkJCQk8L1RSPg0KCQkJCQk8VFI+DQoJCQkJCSA8VEQ+PFNQQU4gY2xhc3M9 ImdlbmVyYWwiPjxJPlV0aWxpemUgb3VyIG1haW5zdHJlYW0gYW5kIGFkdWx0IG5ld3NsZXR0ZXJz DQoJCQkJCSAgdG8gYnJvYWRjYXN0IHlvdXIgbWVzc2FnZSBvdXQgdG8gb3ZlciAxMiBtaWxsaW9u IE9wdC1pbiBNZW1iZXJzLiBQcmljZXMgYXJlDQoJCQkJCSAgYXMgbG93IGFzICQxLjAwIHBlciAx LDAwMCBkZWxpdmVyZWQgZW1haWxzLjwvST48L1NQQU4+PC9URD4NCgkJCQkJPC9UUj4NCgkJCQkJ PFRSPg0KCQkJCQkgPFREPiZuYnNwOzwvVEQ+DQoJCQkJCTwvVFI+DQoJCQkJCTxUUj4NCgkJCQkJ IDxURD48U1BBTiBjbGFzcz0iZ2VuZXJhbCI+PEI+Rm9yIEFkZGl0aW9uYWwgSW5mb3JtYXRpb24g QWJvdXQgUHVyY2hhc2luZw0KCQkJCQkgIEVtYWlsIFRyYWZmaWMgVGhyb3VnaCBBbGxpZWQgSW50 ZXJuZXQgTWFya2V0aW5nIFBsZWFzZSBWaXNpdA0KCQkJCQkgIDxBIEhSRUY9Imh0dHA6Ly9idWxr ZW1haWwuYWxsaWVkbWFya2V0aW5nLm5ldCIgVEFSR0VUPSJfYmxhbmsiPmh0dHA6Ly9idWxrZW1h aWwuYWxsaWVkbWFya2V0aW5nLm5ldDwvQT48L0I+PC9TUEFOPjwvVEQ+DQoJCQkJCTwvVFI+DQoJ CQkJCTxUUj4NCgkJCQkJIDxURD4NCgkJCQkJICAgPEhSPg0KCQkJCQkgPC9URD4NCgkJCQkJPC9U Uj4NCgkJCQkJPFRSPg0KCQkJCQkgPFREPiZuYnNwOzwvVEQ+DQoJCQkJCTwvVFI+DQoJCQkJCTxU Uj4NCgkJCQkJIDxURD48Qj48U1BBTiBjbGFzcz0iZ2VuZXJhbCI+PEJJRz48QklHPjxCSUc+Tm9u LU9wdC1JbiBFbWFpbA0KCQkJCQkgIExpc3RzPC9CSUc+PC9CSUc+PC9CSUc+PC9TUEFOPjwvQj48 L1REPg0KCQkJCQk8L1RSPg0KCQkJCQk8VFI+DQoJCQkJCSA8VEQ+PFNQQU4gY2xhc3M9ImdlbmVy YWwiPjxJPlNwZWNpYWwgUHJpY2luZyBGb3IgRW1haWwgVHJhZmZpYyBUbyBZb3VyIFdlYg0KCQkJ CQkgIFNpdGU6PC9JPjwvU1BBTj48L1REPg0KCQkJCQk8L1RSPg0KCQkJCQk8VFI+DQoJCQkJCSA8 VEQ+Jm5ic3A7PC9URD4NCgkJCQkJPC9UUj4NCgkJCQkJPFRSPg0KCQkJCQkgPFREPjxTUEFOIGNs YXNzPSJnZW5lcmFsIj48Qj5XZSBoYXZlIHZhcmlvdXMgZGlmZmVyZW50IG5vbi1PcHQtSW4gTGlz dHMgZm9yDQoJCQkJCSAgc2FsZS4gVmlzaXQgb3VyIGVtYWlsIHNpdGUgYXQNCgkJCQkJICA8QSBI UkVGPSJodHRwOi8vYnVsa2VtYWlsLmFsbGllZG1hcmtldGluZy5uZXQiIFRBUkdFVD0iX2JsYW5r Ij5odHRwOi8vYnVsa2VtYWlsLmFsbGllZG1hcmtldGluZy5uZXQ8L0E+DQoJCQkJCSAgZm9yIGFk ZGl0aW9uYWwgZGV0YWlscy48L0I+PC9TUEFOPjwvVEQ+DQoJCQkJCTwvVFI+DQoJCQkJCTxUUj4N CgkJCQkJIDxURD4NCgkJCQkJICAgPEhSPg0KCQkJCQkgPC9URD4NCgkJCQkJPC9UUj4NCgkJCQkJ PFRSPg0KCQkJCQkgPFREPiZuYnNwOzwvVEQ+DQoJCQkJCTwvVFI+DQoJCQkJCTxUUj4NCgkJCQkJ IDxURD48Qj48U1BBTiBjbGFzcz0iZ2VuZXJhbCI+PEJJRz48QklHPjxCSUc+U0VDVVJFDQoJCQkJ CSAgRU1BSUxJTkc8L0JJRz48L0JJRz48L0JJRz48L1NQQU4+PC9CPjwvVEQ+DQoJCQkJCTwvVFI+ DQoJCQkJCTxUUj4NCgkJCQkJIDxURD48U1BBTiBjbGFzcz0iZ2VuZXJhbCI+RG8geW91IGhhdmUg YSBsaXN0IGFuZCBuZWVkIHVzIHRvIHNlbmQgb3V0IHlvdXINCgkJCQkJICBlbWFpbHMgdGhyb3Vn aCBvdXIgc2VydmVyPyBObyBwcm9ibGVtISBXZSBjYW4gZG8gaXQgZm9yIG9ubHkgNTAgY2VudHMg cGVyDQoJCQkJCSAgMSwwMDAuPC9TUEFOPjwvVEQ+DQoJCQkJCTwvVFI+DQoJCQkJCTxUUj4NCgkJ CQkJIDxURD4mbmJzcDs8L1REPg0KCQkJCQk8L1RSPg0KCQkJCQk8VFI+DQoJCQkJCSA8VEQ+PFNQ QU4gY2xhc3M9ImdlbmVyYWwiPjxCPmNhbGwgdXMgYXQgOTczLjk5Mi4zOTg1IG9yIGVtYWlsDQoJ CQkJCSAgPEEgSFJFRj0ibWFpbHRvOnRyYWZmaWNAYWxsaWVkbWFya2V0aW5nLm5ldCI+dHJhZmZp Y0BhbGxpZWRtYXJrZXRpbmcubmV0PC9BPjwvQj48L1NQQU4+PC9URD4NCgkJCQkJPC9UUj4NCgkJ CQkJPFRSPg0KCQkJCQkgPFREPg0KCQkJCQkgICA8SFI+DQoJCQkJCSA8L1REPg0KCQkJCQk8L1RS Pg0KCQkJCQk8VFI+DQoJCQkJCSA8VEQ+Jm5ic3A7PC9URD4NCgkJCQkJPC9UUj4NCgkJCQkJPFRS Pg0KCQkJCQkgPFREPjxCPjxTUEFOIGNsYXNzPSJnZW5lcmFsIj48QklHPjxCSUc+PEJJRz5GdXR1 cmVIaXRzIFRyYWZmaWMgLSA0DQoJCQkJCSAgRlJFRSE8L0JJRz48L0JJRz48L0JJRz48L1NQQU4+ PC9CPjwvVEQ+DQoJCQkJCTwvVFI+DQoJCQkJCTxUUj4NCgkJCQkJIDxURD48U1BBTiBjbGFzcz0i Z2VuZXJhbCI+RnV0dXJlSGl0cyBpcyBhIHNvZnR3YXJlIHByb2dyYW0gaG9zdGVkIG9uIG91cg0K CQkJCQkgIHNlcnZlciB0aGF0Jm5ic3A7cGxhY2VzIGEgJm5ic3A7SUNPTiBvbiB5b3VyIHVzZXJz IGRlc2t0b3AsIGZhdm9yaXRlcywNCgkJCQkJICBsaW5rcywmbmJzcDtzdGFydCBidXR0b24gYW5k IGNoYW5nZXMgdGhlaXIgaG9tZXBhZ2UuIFRoaXMgaXMgZG9uZSBlaXRoZXINCgkJCQkJICBhdXRv bWF0aWNhbGx5IG9yIHdpdGggYWxlcnQuIFRoaXMgcHJvZHVjdCBpcyBmcmVlIGFuZCBpcyBsb2Nh dGVkIG9uIG91cg0KCQkJCQkgIDxBIEhSRUY9Imh0dHA6Ly93d3cuYWxsaWVkbWFya2V0aW5nLm5l dCI+Y29ycG9yYXRlIHdlYiBzaXRlPC9BPiBvciBvbg0KCQkJCQkgIDxBIEhSRUY9Imh0dHA6Ly9m dXR1cmVoaXRzLmFsbGllZG1hcmtldGluZy5uZXQiPmh0dHA6Ly9mdXR1cmVoaXRzLmFsbGllZG1h cmtldGluZy5uZXQ8L0E+PC9TUEFOPjwvVEQ+DQoJCQkJCTwvVFI+DQoJCQkJCTxUUj4NCgkJCQkJ IDxURD4mbmJzcDs8L1REPg0KCQkJCQk8L1RSPg0KCQkJCSAgICAgIDwvVEFCTEU+DQoJCQkJICAg IDwvVEQ+DQoJCQkJICA8L1RSPg0KCQkJCTwvVEFCTEU+DQoJCQkgICAgICA8L1REPg0KCQkJICAg IDwvVFI+DQoJCQkgICAgPFRSPg0KCQkJICAgICAgPFREPg0KCQkJCSAgPEhSPg0KCQkJICAgICAg PC9URD4NCgkJCSAgICA8L1RSPg0KCQkJICAgIDxUUj4NCgkJCSAgICAgIDxURD48U1BBTiBjbGFz cz0iZ2VuZXJhbCI+VG8gcmVtb3ZlIHlvdXIgZW1haWwgYWRkcmVzcyBmcm9tIHRoaXMgbGlzdCBh bmQNCgkJCQlhbnkgb3RoZXIgbGlzdHMgYXNzb2NpYXRlZCB0byBUaGUtRW1haWwtSW5mb3JtYXRv cnkNCgkJCQk8QSBIUkVGPSJodHRwOi8vYWRzZXJ2ZXIuY3liZXJzdWJzY3JpYmVyLmNvbS9yZW1v dmUuaHRtbCI+Q0xJQ0sNCgkJCQlIRVJFPC9BPjwvU1BBTj48L1REPg0KCQkJICAgIDwvVFI+DQoJ CQkgIDwvVEFCTEU+DQoJCQk8L1REPg0KCQkgICAgICA8L1RSPg0KCQkgICAgPC9UQUJMRT4NCgkJ ICAgIDxQPg0KCQkgIDwvVEQ+DQoJCTwvVFI+DQoJICAgICAgPC9UQUJMRT4NCgkgICAgPC9URD4N CgkgIDwvVFI+DQoJPC9UQUJMRT4NCiAgICAgIDwvVEQ+DQogICAgPC9UUj4NCiAgPC9UQUJMRT4N CjwvQ0VOVEVSPg0KPFA+DQo8L0JPRFk+PC9IVE1MPg0K From fdrake@acm.org Sat Apr 13 03:07:25 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 12 Apr 2002 22:07:25 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <15543.37452.98720.899406@anthem.wooz.org> References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <15543.30585.922471.967802@12-248-41-177.client.attbi.com> <15543.37452.98720.899406@anthem.wooz.org> Message-ID: <15543.37597.235553.411149@grendel.zope.com> Barry A. Warsaw writes: > I really think we should expose Optik's main parser class through > getopt, which we'd make into a package. I didn't see any response to > my previous suggestion, does everybody else hate it? ;) I think this would be fine, but if the class is OptionParser, I wouldn't rename it to Parser as you suggested. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From Administrator@python.org Sat Apr 13 03:03:49 2002 From: Administrator@python.org (Administrator@python.org) Date: Fri, 12 Apr 2002 22:03:49 -0400 Subject: [Python-Dev] ScanMail Message: To Recipient Match eManager setting and take action. Message-ID: <009a01c1e28f$73c18e70$08cd58c7@basistech.net> **************** eManager Notification ***************** The following mail was blocked since it contains sensitive content. Source mailbox: Thanks4asking@alliedmarketing.net Destination mailbox(es): python-dev@python.org Rule/Policy: Anti-Spam Action: Quarantine to D:\Mail Rule Quarantine\2002-04-12\22-03-49.6344 [Suspected spam] ******************* End of message ********************* From barry@zope.com Sat Apr 13 03:09:42 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 12 Apr 2002 22:09:42 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <15543.30585.922471.967802@12-248-41-177.client.attbi.com> <15543.37452.98720.899406@anthem.wooz.org> <15543.37597.235553.411149@grendel.zope.com> Message-ID: <15543.37734.890896.654268@anthem.wooz.org> >>>>> "Fred" == Fred L Drake, Jr writes: >> I really think we should expose Optik's main parser class >> through getopt, which we'd make into a package. I didn't see >> any response to my previous suggestion, does everybody else >> hate it? ;) Fred> I think this would be fine, but if the class is Fred> OptionParser, I wouldn't rename it to Parser as you Fred> suggested. You're right, that's probably best. -Barry From jacobs@penguin.theopalgroup.com Sat Apr 13 03:15:48 2002 From: jacobs@penguin.theopalgroup.com (Kevin Jacobs) Date: Fri, 12 Apr 2002 22:15:48 -0400 (EDT) Subject: [Python-Dev] trace.py and the obscurity of Tools/scripts/ (was: Unittest list) In-Reply-To: <15543.29059.782535.860209@12-248-41-177.client.attbi.com> Message-ID: On Fri, 12 Apr 2002, Skip Montanaro wrote: > SM> It's speed cries out for a rewrite of some sort. I haven't thought > SM> about it, but I wonder if it could be layered on top of hotshot. > > Jeremy> Can any of the handler methods be re-coded in C? The hotshot > Jeremy> changes allow you to install a C function as a trace hook, but > Jeremy> doesn't the trace function in trace.py do a fair amount of work? > > Dunno. As you might guess I haven't looked at the trace code in quite > awhile... :-) One way to make the function tracking code faster is to use the profiler hooks instead of the tracer hooks. I've already made this and many othe rcleanups nad bug fixes on my local version. I'm also looking at translating bits into C, though I've been side-tracked by an import related problem that trashes module paths. -Kevin -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com From tim.one@comcast.net Sat Apr 13 03:36:23 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 12 Apr 2002 22:36:23 -0400 Subject: [Python-Dev] A "new" kind of leak In-Reply-To: <200204130021.g3D0LZc19139@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > You can get the same effect with sys._getframe() instead of > inspect.currentframe(). Sure. > ... > If Neil won't, someone else can do it. Adding GC to an object isn't > hard, and the tp_traverse/tp_clear implementations probably port right > over. As Neil said, Everything Was Different in 2.1, and frames are especially delicate. I'll add that I expect adding traceback objects to GC in 2.1 too would plug a lot more real-life leaks than adding frameobjects alone. And then there are bound method objects, and ... Neil did a lot of work here for 2.2. >> Question #3: Can we use pymalloc to manage frame objects instead? > How would it help? pymalloc is generally faster than platform malloc/free. >> Question #4: Can we just get rid of the free list? I don't think so >> -- a frame object is needed on every function invocation, and saving >> malloc/free overhead is a real win. > Yup. Try it though. Here's a one-liner patch that's close enough for timing purposes. But please note that the relative speed of such changes varies widely across platforms, and the more we fiddle, the greater the odds of creating a real problem for someone (also of creating a real benefit for someone -- but nobody will remember that). I'm rarely hesitant to try stuff like this in current CVS, because developers are beating on the head daily, across a variety of platforms. Changes checked in to a maintenance branch are likely to get tested for the second time in their life by Python's end users. Index: frameobject.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Objects/frameobject.c,v retrieving revision 2.60 diff -c -r2.60 frameobject.c *** frameobject.c 28 Mar 2002 20:34:59 -0000 2.60 --- frameobject.c 13 Apr 2002 02:09:01 -0000 *************** *** 91,98 **** Py_XDECREF(f->f_exc_type); Py_XDECREF(f->f_exc_value); Py_XDECREF(f->f_exc_traceback); ! f->f_back = free_list; ! free_list = f; Py_TRASHCAN_SAFE_END(f) } --- 91,97 ---- Py_XDECREF(f->f_exc_type); Py_XDECREF(f->f_exc_value); Py_XDECREF(f->f_exc_traceback); ! PyObject_GC_Del(f); Py_TRASHCAN_SAFE_END(f) } >> An effective solution would be to bound the size of the frameobject >> free list ... > Good idea. It's something tuples already do in their free lists. Do we have any other gc types using free lists? I expect the same kind of leak could be provoked wrt any gc container using an unbounded free list. >> Question #5: What's a good value for N? [1000] >> ... > I bet even lowering it to 50 would work just as well in practice. For most programs, yes, although I expect Zope routinely gets 50 levels deep, so would prefer the geometric mean : >>> math.sqrt(1000 * 50) 223.60679774997897 >>> I'll stay up all night deciding whether 223 or 224 is better. > I'm somewhat wary of solutions that use a really high bound for cases > like this; the code that deals with exceeding the bound would almost > never get executed, so bugs there would linger forever. Good point. >> Question #6: If we agree it's a good idea to put on a bound in 2.3, >> is it a good idea to backport the bound? I think yes to 2.2, but no >> to 2.1. In 2.1 the frame-cycle cases leak anyway. > But it was reported for 2.1, right? Yes, but it was reported as a request to enhance the docs. I'm not sure how, and asked the OP to elaborate (but haven't yet heard back). > I could imagine making this a showcase of the new religion: more > support for older versions. 2.1 is the limit though. I'm afraid it may also serve as a showcase for how a one-hour bugfix can spread to consume days. Cut this one off at 2.2 instead and it's not a lot of extra work. From mhammond@skippinet.com.au Sat Apr 13 03:48:08 2002 From: mhammond@skippinet.com.au (Mark Hammond) Date: Sat, 13 Apr 2002 12:48:08 +1000 Subject: [Python-Dev] Python sidebar for Mozilla/Netscape In-Reply-To: Message-ID: Hi all, I updated my Mozilla sidebar. It now supports searching either Python.org search, ActiveState mailing list archive, or Thomas Heller's index. The latter is very cute - instant access to almost anything you need to know :) If you already installed it, just open it and it should be updated. If you haven't yet installed it, see below for the URL. I wont post further updates here, but will make them to comp.lang.python (as the sidebar is growing into a general purpose Python tool, rather than focussed at python-dev) Mark. > Hi all, > I wrote a cute little Python sidebar for Mozilla/NS6. It allows quick > access to bugs/patches and PEPs, and has a search facility using the > ActiveState mailinglist archive. It is pretty cute, but > suggestions for new > features welcome. As it has the PEP and Bugs feature, it is really suited > more to python-dev than python-list - hence my mail here :) > > Check it out at http://starship.python.net/crew/mhammond/mozilla From skip@pobox.com Sat Apr 13 03:56:34 2002 From: skip@pobox.com (Skip Montanaro) Date: Fri, 12 Apr 2002 21:56:34 -0500 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <15543.37452.98720.899406@anthem.wooz.org> References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <15543.30585.922471.967802@12-248-41-177.client.attbi.com> <15543.37452.98720.899406@anthem.wooz.org> Message-ID: <15543.40546.371491.155932@12-248-41-177.client.attbi.com> SM> How about "optionparser" or "optionlib"? BAW> I really think we should expose Optik's main parser class through BAW> getopt, which we'd make into a package. I didn't see any response BAW> to my previous suggestion, does everybody else hate it? ;) I assume this can be done without changing the apparent interface to the getopt class? That is, if I import getopt will getopt.getopt still reference what getopt.getopt does today? Skip From barry@zope.com Sat Apr 13 04:08:43 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 12 Apr 2002 23:08:43 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <15543.30585.922471.967802@12-248-41-177.client.attbi.com> <15543.37452.98720.899406@anthem.wooz.org> <15543.40546.371491.155932@12-248-41-177.client.attbi.com> Message-ID: <15543.41275.421891.469392@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: SM> How about "optionparser" or "optionlib"? BAW> I really think we should expose Optik's main parser class BAW> through getopt, which we'd make into a package. I didn't see BAW> any response to my previous suggestion, does everybody else BAW> hate it? ;) SM> I assume this can be done without changing the apparent SM> interface to the getopt class? That is, if I import getopt SM> will getopt.getopt still reference what getopt.getopt does SM> today? Yes. Move getopt.py to getopt/_getopt.py and put this in getopt/__init__.py: # For backwards compatibility from _getopt import getopt, error, GetoptError # For convenience from OptionParser import OptionParser so some such similar incantation (I don't remember off-hand whether the module names should be fully-qualified or not). -Barry From fdrake@acm.org Sat Apr 13 04:14:26 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 12 Apr 2002 23:14:26 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <15543.41275.421891.469392@anthem.wooz.org> References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <15543.30585.922471.967802@12-248-41-177.client.attbi.com> <15543.37452.98720.899406@anthem.wooz.org> <15543.40546.371491.155932@12-248-41-177.client.attbi.com> <15543.41275.421891.469392@anthem.wooz.org> Message-ID: <15543.41618.485683.441472@grendel.zope.com> Barry A. Warsaw writes: > Yes. Move getopt.py to getopt/_getopt.py and put this in > getopt/__init__.py: Actually, I'd call getopt.py getopt/classic.py, but that's just me. ;) > # For backwards compatibility > from _getopt import getopt, error, GetoptError > # For convenience > from OptionParser import OptionParser > > so some such similar incantation (I don't remember off-hand whether > the module names should be fully-qualified or not). Either should be fine, but we've been recommending full package names for general use, so I think using that would be good form. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From skip@pobox.com Sat Apr 13 04:16:38 2002 From: skip@pobox.com (Skip Montanaro) Date: Fri, 12 Apr 2002 22:16:38 -0500 Subject: [Python-Dev] FAQ 4.102 has been updated - can someone proofread it? In-Reply-To: <200204111522.g3BFMes24367@odiug.zope.com> References: <15541.41731.671262.496339@12-248-41-177.client.attbi.com> <200204111522.g3BFMes24367@odiug.zope.com> Message-ID: <15543.41750.600305.289478@12-248-41-177.client.attbi.com> >> Being the Unicode novice that I am, I'd feel better if someone >> proofread FAQ 4.102: Guido> Hm, I think you've got the ordering wrong. Suggestion #1 would Guido> be to fix the application to do explicit encoding/decoding. Guido> Suggestion #2 would be to change the site default encoding by Guido> adding a sitecustomize.py. I'd leave direct hacking of site.py Guido> out of the suggestions. Taken care of. Thanks for the feedback. Skip From jamescalthoff@yahoo.com Sat Apr 13 06:23:25 2002 From: jamescalthoff@yahoo.com (James Althoff) Date: Fri, 12 Apr 2002 22:23:25 -0700 Subject: [Python-Dev] A "new" kind of leak Message-ID: <000901c1e2ab$571a2220$54dac13f@Pacbell.net> Hey Tim, Remember the exchange below from a while back on python-list (you submitted a nice sorting algorithm in response)? Isn't this the kind of thing that could create lots of generators? I'm not saying one should write this kind of code -- mine was just a "for fun" rewrite of David's original example (he seemed to want to be able to use generators for quicksort) -- and it was relatively slow, in any case. And of course there are lots of other, better ways to get the end result. Just curious. Jim ==================== David Eppstein wrote: >Ok, but your merge routine gets rid of a key advantage of using generators: >it returns a list, rather than yielding the items one at a time, so the >total time is always Theta(n log n), while for the version I posted (and >the cleaner ADT-ized version someone else posted later) the merges all >happen in parallel as items are requested, and the time to pull off the >first k items from a sorted list of n items is O(n + k log n). Ok. Here is essentially the same thing but we return an interator instead of a list (same idea as your version). BTW, I also like Alex's ADT approach. Mine was just a suggestion if class defs were to be avoided. Jim ========================== from __future__ import generators def gen(alist): result = {'isempty':1,'value':None} for value in alist: result['isempty'] = 0 result['value'] = value yield result result['isempty'] = 1 while 1: yield result def merge(list1,list2): gen1, gen2 = gen(list1), gen(list2) next1, next2 = gen1.next(), gen2.next() while 1: if next1['isempty'] and next2['isempty']: break if (next2['isempty'] or (not next1['isempty'] and next1['value'] <= next2['value'])): yield next1['value'] next1 = gen1.next() else: yield next2['value'] next2 = gen2.next() def mergesort(alist): if len(alist) <= 1: return iter(alist) return merge(mergesort(alist[:len(alist)//2]), mergesort(alist[len(alist)//2:])) From tim.one@comcast.net Sat Apr 13 06:48:15 2002 From: tim.one@comcast.net (Tim Peters) Date: Sat, 13 Apr 2002 01:48:15 -0400 Subject: [Python-Dev] A "new" kind of leak In-Reply-To: Message-ID: FYI, I plugged the leak in 2.2 and current CVS via bounding the frameobject free_list. I skipped 2.1 (lots of cyclic stuff leaks there; somebody can tell me to work on that, but I'm not volunteering). I think it took about 4x longer than it would have *just* to fix it in the head. This surprised me. In retrospect, though, it doesn't: obviously, there's twice as much of everything (twice as many checkins to do, twice as many files to edit, twice as many tests to run, etc), and there's no "economy of scale". But it's worse than just that, because there are endless little costs switching between them, and especially on Windows where I use a GUI to build (and have to poke around finding the right project file to load), and changing directories in a DOS box is clumsy, and I also wanted an additional Cygwin shell to try to apply a patch, generated on the trunk, to the 2.2 branch. Trying to switch among all these things while the time-consuming parts are running, in different parts of the process on different branches, quickly leads to mistakes, and then there's extra time to recover from those. There's also an urge to try to copy collateral text (checkin notes, NEWS) from one branch to the other, and so more little files get created to effect the transfer. So it goes -- I didn't think we'd get twice the fixes for free . This was a small bugfix, and so the overheads swamped the time spent just fixing the code. For a larger bugfix I'd expect more like a factor of 3 oerall -- unless the code can't be backported mechanically, in which case it may take arbitrarily longer. predicting-your-future-ly y'rs - tim From niemeyer@conectiva.com Sat Apr 13 07:59:36 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Sat, 13 Apr 2002 03:59:36 -0300 Subject: [Python-Dev] mknod for posixmodule Message-ID: <20020413035936.A2790@ibook.distro.conectiva> People, we need support for mknod() in python mode. This makes possible for python modules like tarfile.py to create special nodes. The first option is of course adding a plain mknod(), with similar parameters to that found in the C interface, like other examples we have in the module. The closest interface would be something like: mknod(filename, mode, major, minor) But the mode parameter, like in the C interface, may be or'ed with S_IFREG, S_IFCHR, S_IFBLK, S_IFIFO. That seems unpythonic. The other option that comes to my mind is: mknod(filename, mode, type, major, minor) That would make more explicit the need for type. OTOH, we already have in the library functions for creating fifos and regular files (S_IFIFO and S_IFREG). So there's another option: mkblkdev(filename, mode, major, minor) mkchrdev(filename, mode, major, minor) What do you think? -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From mwh@python.net Sat Apr 13 11:12:23 2002 From: mwh@python.net (Michael Hudson) Date: 13 Apr 2002 11:12:23 +0100 Subject: [Python-Dev] A "new" kind of leak In-Reply-To: Tim Peters's message of "Sat, 13 Apr 2002 01:48:15 -0400" References: Message-ID: <2mn0w7ykjc.fsf@starship.python.net> Tim Peters writes: > I think it took about 4x longer than it would have *just* to fix it > in the head. This surprised me. In retrospect, though, it doesn't: > obviously, there's twice as much of everything (twice as many > checkins to do, twice as many files to edit, twice as many tests to > run, etc), and there's no "economy of scale". But it's worse than > just that, because there are endless little costs switching between > them, and especially on Windows where I use a GUI to build (and have > to poke around finding the right project file to load), and changing > directories in a DOS box is clumsy, and I also wanted an additional > Cygwin shell to try to apply a patch, generated on the trunk, to the > 2.2 branch. Trying to switch among all these things while the > time-consuming parts are running, in different parts of the process > on different branches, quickly leads to mistakes, and then there's > extra time to recover from those. There's also an urge to try to > copy collateral text (checkin notes, NEWS) from one branch to the > other, and so more little files get created to effect the transfer. > > So it goes -- I didn't think we'd get twice the fixes for free > . This was a small bugfix, and so the overheads swamped the > time spent just fixing the code. For a larger bugfix I'd expect > more like a factor of 3 oerall -- unless the code can't be > backported mechanically, in which case it may take arbitrarily > longer. For patches that cvs up -j blah -j blat file can handle, I have a setup that make porting them the work of seconds. It takes a little while to set up, so I batch them. I don't run the tests for every checkin this way so I guess I risk pain -- if I plough through 20 or so fixes in a run and find somethings broken, working out what did it could be entertaining. Hasn't happened yet, though. What I'm saying is, if getting fixes into the tree is awkward for you, don't worry about it /too/ much. I want to fiddle my scripts a bit, but they should appear in Tools/ at some point... Cheers, M. -- Strangely enough I saw just such a beast at the grocery store last night. Starbucks sells Javachip. (It's ice cream, but that shouldn't be an obstacle for the Java marketing people.) -- Jeremy Hylton, 29 Apr 1997 From mwh@python.net Sat Apr 13 11:16:07 2002 From: mwh@python.net (Michael Hudson) Date: 13 Apr 2002 11:16:07 +0100 Subject: [Python-Dev] mknod for posixmodule In-Reply-To: Gustavo Niemeyer's message of "Sat, 13 Apr 2002 03:59:36 -0300" References: <20020413035936.A2790@ibook.distro.conectiva> Message-ID: <2mk7rbykd4.fsf@starship.python.net> Gustavo Niemeyer writes: > People, > > we need support for mknod() in python mode. This makes possible > for python modules like tarfile.py to create special nodes. I can believe it. > The first option is of course adding a plain mknod(), with similar > parameters to that found in the C interface, like other examples > we have in the module. The closest interface would be something > like: > > mknod(filename, mode, major, minor) > > But the mode parameter, like in the C interface, may be or'ed with > S_IFREG, S_IFCHR, S_IFBLK, S_IFIFO. That seems unpythonic. Generally, I think we wrap this sort of function as thinly as possible. That way you can use the man page as help. And this isn't the sort of function that fragile newbies are likely to trip over is it? Cheers, M. -- I don't remember any dirty green trousers. -- Ian Jackson, ucam.chat From mwh@python.net Sat Apr 13 11:40:15 2002 From: mwh@python.net (Michael Hudson) Date: 13 Apr 2002 11:40:15 +0100 Subject: [Python-Dev] Porting bug fixes (was A "new" kind of leak) In-Reply-To: Michael Hudson's message of "13 Apr 2002 11:12:23 +0100" References: <2mn0w7ykjc.fsf@starship.python.net> Message-ID: <2mhemfyj8w.fsf_-_@starship.python.net> The bits I forgot: Michael Hudson writes: > What I'm saying is, if getting fixes into the tree is awkward for > you, don't worry about it /too/ much. the major downside to this problem is that I need to read python-checkins pretty assiduously. There may be a way around this... thinking aloud, I could subscribe say mwh-busybee@python.net to -checkins which I could get qmail to run through a script that would scan the checkin message for a key phrase ("bugfix" suggests itself) and drop the message on the floor if it doesn't contain the phrase and put it into an appropriate gnus folder if it does... So what I'm suggesting is that if you want a checkin to be ported to release22-maint you should add a specific bit of text to the chickin message. Does this seem reasonable? Another random point: it would be nice if on checking a bugfix into HEAD people *said* if *they* planned to port the fix themselves. Otherwise I see the message that says "bugfix candidate", hit they key that copies it into my special bugfixes folder, then read on and find it's already been ported and have to go and find the copy and delete it. TIA. Cheers, M. -- The "of course, while I have no problem with this at all, it's surely too much for a lesser being" flavor of argument always rings hollow to me. -- Tim Peters, 29 Apr 1998 From martin@v.loewis.de Sat Apr 13 11:40:36 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 13 Apr 2002 12:40:36 +0200 Subject: [Python-Dev] mknod for posixmodule In-Reply-To: <2mk7rbykd4.fsf@starship.python.net> References: <20020413035936.A2790@ibook.distro.conectiva> <2mk7rbykd4.fsf@starship.python.net> Message-ID: Michael Hudson writes: > Generally, I think we wrap this sort of function as thinly as > possible. That way you can use the man page as help. And this isn't > the sort of function that fragile newbies are likely to trip over is > it? I completely agree. Martin From guido@python.org Sat Apr 13 11:47:13 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 13 Apr 2002 06:47:13 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: Your message of "Fri, 12 Apr 2002 22:05:00 EDT." <15543.37452.98720.899406@anthem.wooz.org> References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <15543.30585.922471.967802@12-248-41-177.client.attbi.com> <15543.37452.98720.899406@anthem.wooz.org> Message-ID: <200204131047.g3DAlD520681@pcp742651pcs.reston01.va.comcast.net> > >>>>> "SM" == Skip Montanaro writes: > > SM> How about "optionparser" or "optionlib"? > > I really think we should expose Optik's main parser class through > getopt, which we'd make into a package. I didn't see any response to > my previous suggestion, does everybody else hate it? ;) To the contrary. Only problem is how to prevent loading all of optik when one imports getopt.getopt (and getopt.error). --Guido van Rossum (home page: http://www.python.org/~guido/) From martin@v.loewis.de Sat Apr 13 11:48:23 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 13 Apr 2002 12:48:23 +0200 Subject: [Python-Dev] Porting bug fixes (was A "new" kind of leak) In-Reply-To: <2mhemfyj8w.fsf_-_@starship.python.net> References: <2mn0w7ykjc.fsf@starship.python.net> <2mhemfyj8w.fsf_-_@starship.python.net> Message-ID: Michael Hudson writes: > So what I'm suggesting is that if you want a checkin to be ported to > release22-maint you should add a specific bit of text to the chickin > message. Does this seem reasonable? If you need a specific phrase, just announce that (preferably, document it somewhere also on python.org/dev). > Another random point: it would be nice if on checking a bugfix into > HEAD people *said* if *they* planned to port the fix themselves. > Otherwise I see the message that says "bugfix candidate", hit they key > that copies it into my special bugfixes folder, then read on and find > it's already been ported and have to go and find the copy and delete > it. If I'm going to commit the same patch onto the maintainance branch, I usually don't mark it as "bugfix candidate". Regards, Martin From barry@zope.com Sat Apr 13 14:53:07 2002 From: barry@zope.com (Barry A. Warsaw) Date: Sat, 13 Apr 2002 09:53:07 -0400 Subject: [Python-Dev] mknod for posixmodule References: <20020413035936.A2790@ibook.distro.conectiva> <2mk7rbykd4.fsf@starship.python.net> Message-ID: <15544.14403.391666.693272@anthem.wooz.org> >>>>> "MH" == Michael Hudson writes: MH> Generally, I think we wrap this sort of function as thinly as MH> possible. That way you can use the man page as help. +1 From nas@python.ca Sat Apr 13 15:19:38 2002 From: nas@python.ca (Neil Schemenauer) Date: Sat, 13 Apr 2002 07:19:38 -0700 Subject: [Python-Dev] A "new" kind of leak In-Reply-To: ; from tim.one@comcast.net on Fri, Apr 12, 2002 at 07:40:22PM -0400 References: Message-ID: <20020413071937.A13273@glacier.arctrix.com> PyCFunctionObject and PyMethodObject also use free lists and set the GC flag. Neil From guido@python.org Sat Apr 13 15:21:46 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 13 Apr 2002 10:21:46 -0400 Subject: [Python-Dev] mknod for posixmodule In-Reply-To: Your message of "13 Apr 2002 11:16:07 BST." <2mk7rbykd4.fsf@starship.python.net> References: <20020413035936.A2790@ibook.distro.conectiva> <2mk7rbykd4.fsf@starship.python.net> Message-ID: <200204131421.g3DELka21628@pcp742651pcs.reston01.va.comcast.net> > > mknod(filename, mode, major, minor) > > > > But the mode parameter, like in the C interface, may be or'ed with > > S_IFREG, S_IFCHR, S_IFBLK, S_IFIFO. That seems unpythonic. Not to me. > Generally, I think we wrap this sort of function as thinly as > possible. That way you can use the man page as help. And this isn't > the sort of function that fragile newbies are likely to trip over is > it? Agreed. --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@pythoncraft.com Sat Apr 13 15:40:49 2002 From: aahz@pythoncraft.com (Aahz) Date: Sat, 13 Apr 2002 10:40:49 -0400 Subject: [Python-Dev] Building on Windows (was Re: A "new" kind of leak) In-Reply-To: References: Message-ID: <20020413144048.GA26219@panix.com> On Sat, Apr 13, 2002, Tim Peters wrote: > > I think it took about 4x longer than it would have *just* to fix it in the > head. This surprised me. In retrospect, though, it doesn't: obviously, > there's twice as much of everything (twice as many checkins to do, twice as > many files to edit, twice as many tests to run, etc), and there's no > "economy of scale". But it's worse than just that, because there are > endless little costs switching between them, and especially on Windows where > I use a GUI to build (and have to poke around finding the right project file > to load), and changing directories in a DOS box is clumsy, and I also wanted > an additional Cygwin shell to try to apply a patch, generated on the trunk, > to the 2.2 branch. Trying to switch among all these things while the > time-consuming parts are running, in different parts of the process on > different branches, quickly leads to mistakes, and then there's extra time > to recover from those. There's also an urge to try to copy collateral text > (checkin notes, NEWS) from one branch to the other, and so more little files > get created to effect the transfer. Yup, this is precisely why I never considered trying to do serious dev work on Windows. I think this kind of thing is much more amenable to automation on Unixes, but I also think the NT-class OSes do better, too. Have you considered switching to Win2K for your dev work and just testing under Win98? To what extent do you think you could automate things using Python and WSH? What about using multiple dekstops to keep projects straight? [I'm leaving this on python-dev for the moment, because I think we should all know what it takes to keep Windows viable.] -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ What if there were no rhetorical questions? From niemeyer@conectiva.com Sat Apr 13 18:52:31 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Sat, 13 Apr 2002 14:52:31 -0300 Subject: [Python-Dev] mknod for posixmodule In-Reply-To: <2mk7rbykd4.fsf@starship.python.net> References: <20020413035936.A2790@ibook.distro.conectiva> <2mk7rbykd4.fsf@starship.python.net> Message-ID: <20020413145230.A9383@ibook.distro.conectiva> > Generally, I think we wrap this sort of function as thinly as > possible. That way you can use the man page as help. And this isn't > the sort of function that fragile newbies are likely to trip over is > it? Thanks Michael! Patch #543447 submitted. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From tim.one@comcast.net Sat Apr 13 19:10:08 2002 From: tim.one@comcast.net (Tim Peters) Date: Sat, 13 Apr 2002 14:10:08 -0400 Subject: [Python-Dev] A "new" kind of leak In-Reply-To: <2mn0w7ykjc.fsf@starship.python.net> Message-ID: [Michael Hudson] > For patches that > > cvs up -j blah -j blat file > > can handle, I have a setup that make porting them the work of seconds. > It takes a little while to set up, so I batch them. Maybe waiting for a change to show up in the trunk is a better way to go. Since I was making the trunk change "live", and wasn't going to check anything in before everything worked on both trunk and branch, -j was impotent (in the way I happened to do this). Regardless, it won't work for *this* patch if it's desired in 2.1 (too much has changed). > I don't run the tests for every checkin this way so I guess I risk > pain -- if I plough through 20 or so fixes in a run and find > somethings broken, working out what did it could be entertaining. > Hasn't happened yet, though. In this particular case, we don't have any machinery in the test suite for detecting unbounded process growth, and I don't know a low-effort way to add some that would actually work x-platform. I did have a "by hand" test to run, which runs forever while I stare at the output of a Windows process-size sniffer. So this particular case required running tests manually, and if "it breaks" (in the sense of not clamping process growth) someday, that won't be detected automatically. > What I'm saying is, if getting fixes into the tree is awkward for you, > don't worry about it /too/ much. It would have been easier if I checked in the trunk changes first before bothering with the 2.2 branch. Still, the primary point stands: when writing the code goes fast (as it does with most bugfixes), and if everything is great, it still takes more than N times as long to stick a fix into N releases. So batching 'em up may be an expedient idea, as it certainly allows amortized some of the expenses (it all adds up! and, e.g., doing a cvs up over the network in a branch burns the same amount of time whether you're doing the update in preparation for backporting 1 fix or 1000). > I want to fiddle my scripts a bit, but they should appear in Tools/ at > some point... Cool! From guido@python.org Sat Apr 13 19:18:33 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 13 Apr 2002 14:18:33 -0400 Subject: [Python-Dev] Building on Windows (was Re: A "new" kind of leak) In-Reply-To: Your message of "Sat, 13 Apr 2002 10:40:49 EDT." <20020413144048.GA26219@panix.com> References: <20020413144048.GA26219@panix.com> Message-ID: <200204131818.g3DIIXV21796@pcp742651pcs.reston01.va.comcast.net> > On Sat, Apr 13, 2002, Tim Peters wrote: [A lament on doing a patch on two brances] [Aahz] > Yup, this is precisely why I never considered trying to do serious dev > work on Windows. I think this kind of thing is much more amenable to > automation on Unixes, but I also think the NT-class OSes do better, too. > Have you considered switching to Win2K for your dev work and just testing > under Win98? To what extent do you think you could automate things using > Python and WSH? What about using multiple dekstops to keep projects > straight? Tim has both Win2K vs. Win98, and it makes little difference; he's got the same environment on both. I've seen him work. IMO, the debugger (VC++) and editor (not VC) he uses are much better than anything I've seen on Unix -- but the tools he has to use to interact with CVS are much worse. (He'll probably deny this, so I'll stop. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Sat Apr 13 19:19:30 2002 From: tim.one@comcast.net (Tim Peters) Date: Sat, 13 Apr 2002 14:19:30 -0400 Subject: [Python-Dev] RE: Building on Windows (was Re: A "new" kind of leak) In-Reply-To: <20020413144048.GA26219@panix.com> Message-ID: [Aahz] > Yup, this is precisely why I never considered trying to do serious dev > work on Windows. It's not *that* hard. The worst part of heavy development on Win98 is the time it takes for frequent forced reboots; the rest is comparatively minor. > I think this kind of thing is much more amenable to automation on > Unixes, Especially when the project is primarily developed by Unix weenies . If it were built on COM automation, the Unixoids would be crawling on their bellies. > but I also think the NT-class OSes do better, too. Oh yes. Far fewer reboots, and a much stronger native shell. > Have you considered switching to Win2K for your dev work and just > testing under Win98? I use Win2K on my office box. For home use, I still prefer Win98 for everything except development. I've been toying with setting up a Linux box at home, but the time it would take to get that all hooked up and working, then maintained over time, has so far seemed too depressing to overcome. From tim.one@comcast.net Sat Apr 13 19:29:49 2002 From: tim.one@comcast.net (Tim Peters) Date: Sat, 13 Apr 2002 14:29:49 -0400 Subject: [Python-Dev] Building on Windows (was Re: A "new" kind of leak) In-Reply-To: <200204131818.g3DIIXV21796@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > Tim has both Win2K vs. Win98, and it makes little difference; he's got > the same environment on both. I've seen him work. The huge differences here between 2K and 98 are system stability and native shell power. Both are much better on 2K. > IMO, the debugger (VC++) and editor (not VC) he uses are much better > than anything I've seen on Unix -- Mine too, and I used to be an Emacs wizard. I'm afraid if I ever went back to Unix, I'd spend the first three months adding a massive subsystem to Emacs to get back all the C smarts Source Insight has <0.2 wink -- but I really do fear that!>. > but the tools he has to use to interact with CVS are > much worse. > > (He'll probably deny this, so I'll stop. :-) Nope, I fully agree on all counts, except that Win98 is substantially feebler (than W2K) in two ways that count. There are better CVS tools for Windows, but for some reason I never gave up my command-line junkie nature, despite landing on a system with a spectacularly lame shell. "Doing something about that" has been on my todo list since 1994 <0.5 wink>. From niemeyer@conectiva.com Sat Apr 13 19:57:50 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Sat, 13 Apr 2002 15:57:50 -0300 Subject: [Python-Dev] Re: tarfile In-Reply-To: <1128502796.20020413174605@gustaebel.de> References: <20020413001935.A1501@ibook.distro.conectiva> <1128502796.20020413174605@gustaebel.de> Message-ID: <20020413155750.A9454@ibook.distro.conectiva> Hello again Lars! > During the last weeks, most people who sent feedback on tarfile, told > me I should propose it to the standard library. The last one was > Thomas Heller two days ago. All this really encouraged me and if you > would not have come up to me today, I would have come up to you, > sooner or later... Thank you very much. I'm the one who has to thank. You seem to have done a fairly complete work! > tarfile is now about 7 weeks old and is reaching a stable state. But > there are still some things to be done: Nice! I've seen some of your announcements on c.l.py > - It is yet missing intense testing, so I'd like to make another > announcement on c.l.p. No problems about that. I think that some unittests could do part of that work as well. Once it gets into the standard library it will be exposed and may be extensively tested. > - The interface is not yet finally decided. I'm not absolutely > confident of the naming of class TarFile's read(), readstr(), write() > and writestr() methods. They create the illusion of using TarFile as a > file-like object, which it is not. Perhaps they should be named more > in accordance to GNU tar. I'd rather choose a default interface similar where possible to that of zipfile. This makes it easy to include support to tar.gz in programs already supporting zipfile, and also creates some standardization. I'm including python-dev in the discussion to know what they think about this. > - The source code has to be cleaned and the docstrings have to be set > up-to-date. I can help you in any of those tasks, if you need. Some issues: I've just submitted a patch to include mknod() in posixmodule. This will make it easy to include the missing support for special device files. You have implemented the most important GNU extensions (long filenames), and mention you'd like to implement others. OTOH, I've read in the tar documentation that some of the extensions may change until tar 2.0. This would turn into a bad option implementing them right now, IMO. What's your opinion about this? You mention in some document that one of the differences between tarfile and tar is that tarfile uses relative path names. Isn't that the default behavior of tar as well, which could be changed by the -P parameter? > You see, your mail messed up my schedule a bit :-) Don't worry.. we're not in a hurry. :-) > I'm currently working on a latex version of the documentation and will > soon start writing a PEP. That's nice! > Well, I can only say that it would be an honour to give tarfile to the > standard library and there's nothing from my side which would stand in > the way. Thank you very much! -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From niemeyer@conectiva.com Sat Apr 13 20:28:44 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Sat, 13 Apr 2002 16:28:44 -0300 Subject: [Python-Dev] Porting bug fixes (was A "new" kind of leak) In-Reply-To: <2mhemfyj8w.fsf_-_@starship.python.net> References: <2mn0w7ykjc.fsf@starship.python.net> <2mhemfyj8w.fsf_-_@starship.python.net> Message-ID: <20020413162844.A9685@ibook.distro.conectiva> Hi Michael! > There may be a way around this... thinking aloud, I could subscribe > say mwh-busybee@python.net to -checkins which I could get qmail to run > through a script that would scan the checkin message for a key phrase > ("bugfix" suggests itself) and drop the message on the floor if it > doesn't contain the phrase and put it into an appropriate gnus folder > if it does... Helping you to think aloud , these lines in .procmailrc are enough for this: :0 B * !bugfix /dev/null :0 bugfixes I'm just not sure how gnus store its mail. If it's a maildir, you have append a "/" to the "bugfixes" above, or "/.", if it's MH. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From tim.one@comcast.net Sat Apr 13 21:49:18 2002 From: tim.one@comcast.net (Tim Peters) Date: Sat, 13 Apr 2002 16:49:18 -0400 Subject: [Python-Dev] Porting bug fixes (was A "new" kind of leak) In-Reply-To: <2mhemfyj8w.fsf_-_@starship.python.net> Message-ID: [Michael Hudson] > ... > So what I'm suggesting is that if you want a checkin to be ported to > release22-maint you should add a specific bit of text to the chickin > message. Does this seem reasonable? Yes, but you have to Pronounce on which specific bit of text you want to see. It's going to get much more complicated if we intend to backport fixes across 2 or 3 years of older releases. I predict that's not going to work unless we establish an easy-to-update patch database recording which patches have and haven't been applied to each old release, which should and shouldn't be applied to each old release, and everyone is serious about keeping that up to date. I'm not aware of any commerical organizations with full-time QA departments that sign up for something so messy, and I'm not sanguine about our prospects of pulling it off (the older the code, the more likely "a bugfix" is to create at least as many problems as it solves; and the more active branches, the more likely fixes to get dropped on the floor). > Another random point: it would be nice if on checking a bugfix into > HEAD people *said* if *they* planned to port the fix themselves. > Otherwise I see the message that says "bugfix candidate", hit they key > that copies it into my special bugfixes folder, then read on and find > it's already been ported and have to go and find the copy and delete > it. TIA. I already do all that stuff, so stop yelling at me . [Martin v. Loewis] > If I'm going to commit the same patch onto the maintainance branch, I > usually don't mark it as "bugfix candidate". Except that "the" maintenance branch loses clear meaning when there are multiple maintenance branches. That's why I expect this just isn't going to work without a patch database: it needs something independent of scattered checkin messages to correlate a *conceptual* patch with all the active branches. Or it needs a truly dedicated person to sign up for each active branch, who actively worries about every patch that comes by. I expect Neil spoke for most current developers there: they don't fear current releases, so won't volunteer for such work (there's no payback for them -- open source works because developers and users volunteer to scratch their own *current* itches, and share the relief; the "maintenance branch" business is unique in that nobody with that particular itch has volunteered to do anything to scratch it). From nhodgson@bigpond.net.au Sat Apr 13 22:35:26 2002 From: nhodgson@bigpond.net.au (Neil Hodgson) Date: Sun, 14 Apr 2002 07:35:26 +1000 Subject: [Python-Dev] A "new" kind of leak References: Message-ID: <005f01c1e333$20299d80$0acc8490@neil> Tim Peters: > I think it took about 4x longer than it would have *just* to fix it in the > head. This surprised me. In retrospect, though, it doesn't: obviously, > there's twice as much of everything (twice as many checkins to do, twice as > many files to edit, twice as many tests to run, etc), and there's no > "economy of scale". But it's worse than just that, because there are > endless little costs switching between them, and especially on Windows where > I use a GUI to build (and have to poke around finding the right project file > to load), and changing directories in a DOS box is clumsy, and I also wanted > ... Are you aware of the ability to build Visual C++ projects from the command line just like good old make files? Like this: msdev SciLexer.dsp /MAKE "SciLexer - Win32 Release" /REBUILD Whenever I finish a significant change to my code base I run a batch script that builds 7 different targets with 3 different compilers. If this had to run up the MSVC GUI for the 3 MSVC projects I'd never bother. For documentation, search for "useenv" in the on-line help. Also works for workspaces. Windows developers do put up with a lot but there is a limit. Neil From guido@python.org Sun Apr 14 00:01:06 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 13 Apr 2002 19:01:06 -0400 Subject: [Python-Dev] Porting bug fixes (was A "new" kind of leak) In-Reply-To: Your message of "Sat, 13 Apr 2002 16:49:18 EDT." References: Message-ID: <200204132301.g3DN16p22095@pcp742651pcs.reston01.va.comcast.net> > Yes, but you have to Pronounce on which specific bit of text you > want to see. It's going to get much more complicated if we intend > to backport fixes across 2 or 3 years of older releases. I predict > that's not going to work unless we establish an easy-to-update patch > database recording which patches have and haven't been applied to > each old release, which should and shouldn't be applied to each old > release, and everyone is serious about keeping that up to date. I'm > not aware of any commerical organizations with full-time QA > departments that sign up for something so messy, and I'm not > sanguine about our prospects of pulling it off (the older the code, > the more likely "a bugfix" is to create at least as many problems as > it solves; and the more active branches, the more likely fixes to > get dropped on the floor). I've got some half-working tools for working with CVS from Python. Maybe it's time to dust those off and write our own database... > [Martin v. Loewis] > > If I'm going to commit the same patch onto the maintainance branch, I > > usually don't mark it as "bugfix candidate". > > Except that "the" maintenance branch loses clear meaning when there > are multiple maintenance branches. That's why I expect this just > isn't going to work without a patch database: it needs something > independent of scattered checkin messages to correlate a > *conceptual* patch with all the active branches. Or it needs a > truly dedicated person to sign up for each active branch, who > actively worries about every patch that comes by. I expect Neil > spoke for most current developers there: they don't fear current > releases, so won't volunteer for such work (there's no payback for > them -- open source works because developers and users volunteer to > scratch their own *current* itches, and share the relief; the > "maintenance branch" business is unique in that nobody with that > particular itch has volunteered to do anything to scratch it). There are several developers who also have a Python-based business with customers who want stable releases. They should join forces and offer some help here. If they don't, I'm not sure I can afford to be very sympathetic to their cause (well, I can show sympathy anyway, but I won't be able to do anything about it :-). --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Sun Apr 14 00:02:22 2002 From: tim.one@comcast.net (Tim Peters) Date: Sat, 13 Apr 2002 19:02:22 -0400 Subject: [Python-Dev] A "new" kind of leak In-Reply-To: <000901c1e2ab$571a2220$54dac13f@Pacbell.net> Message-ID: [James Althoff] > Remember the exchange below from a while back on python-list (you > submitted a nice sorting algorithm in response)? Yup! > Isn't this the kind of thing that could create lots of > generators? Sure -- it creates about as many generators as there are elements in the list being sorted. They're all alive simultaneously, though, so the size of the frame free list doesn't come into play except across distinct top-level sorts. Code like that continues to work fine -- it's not a limit on the number of frames (or generators) you can have, it's only a limit on the amount of memory *permanently* devoted to holding frames (and nothing but frames, and even when no frames are actually in use anymore -- once a blob of memory gets on a type-specific free list, it stays there until Python shuts down even if it's never used again, and it can't be reused for anything except an object of the same type; pymalloc at least allows for reuse based on the number of bytes needed independent of type)). > I'm not saying one should write this kind of code -- mine was just > a "for fun" rewrite of David's original example (he seemed to want > to be able to use generators for quicksort) -- and it was relatively > slow, in any case. And of course there are lots of other, better > ways to get the end result. No need to make excuses . It's a very slick way to get the k smallest elements in worst-case O(n + k log n) time, and instructive for that reason alone. For large enough n and small enough k, it would beat the pants off peeling the first k elements off the result of doing list.sort(), but the overhead is so high it might take a value of n so large that you run out of RAM to hold frames. Many of the frames in this case stay alive a long time, and the malloc/free overhead becomes less significant the more work a frame does over its lifetime. The frame free list is aimed more at optimizing def plus1(n): return n+1 and other trivial functions and methods. There a malloc likely costs more than executing the entire body of the function. From guido@python.org Sun Apr 14 00:54:46 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 13 Apr 2002 19:54:46 -0400 Subject: [Python-Dev] PEP 278 - finished? Message-ID: <200204132354.g3DNskI22181@pcp742651pcs.reston01.va.comcast.net> There were a few comments on Jack's PEP, but by and large they were positive. I've given Jack approval to check his code in (of course it'll have to pass the unit tests). I expect that there will be some work on it between his checkin and the 2.3 release: - maybe it's unnecessary to have a configure option for this - maybe it's better to have a separate constructor instead of a mode argument But those changes are easily made once the code is checked in. So, take it away, Jack! --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Sun Apr 14 03:53:11 2002 From: tim.one@comcast.net (Tim Peters) Date: Sat, 13 Apr 2002 22:53:11 -0400 Subject: [Python-Dev] A "new" kind of leak In-Reply-To: <005f01c1e333$20299d80$0acc8490@neil> Message-ID: [Neil Hodgson] > Are you aware of the ability to build Visual C++ projects from the > command line just like good old make files? Like this: > > msdev SciLexer.dsp /MAKE "SciLexer - Win32 Release" /REBUILD Dimly, from a previous life. Hitting "Batch Build" once in the GUI manages to build 18 Python targets (DLLs and exes), once each in two configurations. Looks like \msdev\msdev98\bin\msdev pcbuild.dsw /make ALL from within the PCbuild directory is equivalent. Cool. The first time I get a warning msg from a Unix checkin and have to bring up the GUI anyway to stare at it in context is the last time I'll use it . From jafo@tummy.com Sun Apr 14 05:57:09 2002 From: jafo@tummy.com (Sean Reifschneider) Date: Sat, 13 Apr 2002 22:57:09 -0600 Subject: [Python-Dev] Re: RELEASED: Python 2.1.3 In-Reply-To: <200204090255.g392tex17317@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Mon, Apr 08, 2002 at 10:55:40PM -0400 References: <200204090255.g392tex17317@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020413225709.B18327@tummy.com> On Mon, Apr 08, 2002 at 10:55:40PM -0400, Guido van Rossum wrote: >Today, April 8th 2002, we've released Python 2.1.3, the latest version I have recently put up RPMs for 2.1.3, sources and bnaries compiled against Red Hat 7.2. See: http://www.python.org/2.1.3/rpms.html or ftp://ftp.python.org/pub/python/2.1.3/rpms/ for more information. Sean -- Do bad programmers wake up on Christmas morning to find coal in their sockets? -- Sean Reifschneider Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From andy@reportlab.com Sun Apr 14 06:42:14 2002 From: andy@reportlab.com (Andy Robinson) Date: Sun, 14 Apr 2002 06:42:14 +0100 Subject: [Python-Dev] Building on Windows (was Re: A "new" kind of leak) In-Reply-To: <20020413141701.3333.69927.Mailman@mail.python.org> Message-ID: > changing directories in a DOS box is clumsy, and I You do know how to turn on command line completion? ;-) HKCU | HKLM -> Software -> Microsoft -> Command Processor -> CompletionChar -> 0x00009 and then the tab key completes. (You may need to restart any current DOS boxes) This works on Win2k and NT4. There are better shells available for Win98 somewhere, but the NT/2000 batch language is pretty capable. Not as well understood, but you can automate lots of processes. Not that I dispute yur basic thesis - we've managed to avoid branching at ReportLab to date for exactly this reason :-) - Andy Robinson From Norman Vine" Andy Robinson writes: >There are better shells available for Win98 somewhere, OpenSource replacment for cmd.exe that runs on WIn9x, WinNT, Win2K, WinME http://www.neuro.gatech.edu/users/cwilson/cygutils/cmd-reactos/index.html and there is always Cygwin :-) Cheers Norman From skip@mojam.com Sun Apr 14 13:00:24 2002 From: skip@mojam.com (Skip Montanaro) Date: Sun, 14 Apr 2002 07:00:24 -0500 Subject: [Python-Dev] Weekly Python Bug/Patch Summary Message-ID: <200204141200.g3EC0OC01992@12-248-41-177.client.attbi.com> Bug/Patch Summary ----------------- 235 open / 2403 total bugs (+9) 93 open / 1421 total patches (-4) New Bugs -------- test_email fails on Mac in summer (2002-04-07) http://python.org/sf/540720 PyType_GenericNew broken (2002-04-08) http://python.org/sf/540965 link to index.html from modindex.html (2002-04-08) http://python.org/sf/541257 IDLE can clobber files (2002-04-09) http://python.org/sf/541730 delattr() causes python to segfault (2002-04-09) http://python.org/sf/541883 urllib.urlopen throws OSError (2002-04-10) http://python.org/sf/541980 Compile fails on posixmodule.c (2002-04-10) http://python.org/sf/542003 long file name support broken in windows (2002-04-10) http://python.org/sf/542314 Problem compiling Python (2002-04-11) http://python.org/sf/542482 Build unable to import w/gcc 3.0.4 (2002-04-11) http://python.org/sf/542737 Config option --without-pth not working? (2002-04-11) http://python.org/sf/542751 Distutils readme outdated (2002-04-12) http://python.org/sf/542912 allow use of descriptors for __doc_ attr (2002-04-12) http://python.org/sf/542984 Memory leak with stackframes + inspect (2002-04-12) http://python.org/sf/543148 installation atop 2.2 fails (2002-04-12) http://python.org/sf/543244 SocketServer initializing outbound UDP (2002-04-12) http://python.org/sf/543318 Interpreter crashes when recoding (2002-04-13) http://python.org/sf/543344 bug? floor divison on complex (2002-04-13) http://python.org/sf/543387 pdb: contradictory msg for "exit"/Ctrl-D (2002-04-14) http://python.org/sf/543674 New Patches ----------- context sensitive help/keyword search (2002-04-08) http://python.org/sf/541031 whichdb unittest (2002-04-09) http://python.org/sf/541694 clean up trace.py (2002-04-11) http://python.org/sf/542562 start docs for PyEval_* functions (2002-04-12) http://python.org/sf/543098 s/Copyright/License/ in bdist_rpm.py (2002-04-13) http://python.org/sf/543498 Closed Bugs ----------- Missing docs for module knee (2002-02-10) http://python.org/sf/515745 Undocumented Py_InitModule (2002-02-22) http://python.org/sf/521448 USE_CACHE_ALIGNED still helpful? (2002-02-28) http://python.org/sf/524062 unicodeobject can coredump on exit (2002-03-11) http://python.org/sf/528620 configure.in Assumes cc_r Exists on AIX. (2002-03-21) http://python.org/sf/533188 Python open w/ MSVC6: bad error msgs (2002-04-03) http://python.org/sf/538827 Closed Patches -------------- threaded profiler. (2001-02-23) http://python.org/sf/403972 patches for python-mode.el V4.1 (2001-04-23) http://python.org/sf/418465 Improved modulefinder (2001-09-19) http://python.org/sf/462936 getrusage() returns struct-like object. (2002-02-01) http://python.org/sf/512005 Finding "home" in "user.py" for Windows (2002-02-27) http://python.org/sf/523424 Unicode fix for test in tkFileDialog.py (2002-04-04) http://python.org/sf/539392 build info docs from sources (2002-04-04) http://python.org/sf/539486 build info docs from tex sources (2002-04-04) http://python.org/sf/539487 dict.popitem(key=None) (2002-04-05) http://python.org/sf/539949 From loewis@informatik.hu-berlin.de Sun Apr 14 13:21:34 2002 From: loewis@informatik.hu-berlin.de (Martin v. =?iso-8859-1?q?L=F6wis?=) Date: 14 Apr 2002 14:21:34 +0200 Subject: [Python-Dev] New autoconf in use! In-Reply-To: <15543.27333.576972.493970@12-248-41-177.client.attbi.com> References: <200204111448.g3BEmQP25970@pcp742651pcs.reston01.va.comcast.net> <200204111545.g3BFj6v24771@odiug.zope.com> <200204121642.g3CGgj010545@pcp742651pcs.reston01.va.comcast.net> <15543.27333.576972.493970@12-248-41-177.client.attbi.com> Message-ID: Skip Montanaro writes: > Agreed. Most machines are fast enough nowadays that the absence of > config.cache isn't a major stumbling block. If they are that > sensitive to the performance reduction, they can install autoconf > themselves and generate a local version of the configure script. I found that the most convenient way to get configure caching is to define a variable CONFIG_SITE pointing to a file that reads if test "$cache_file" = /dev/null; then cache_file="$HOME/.ac/config.cache" fi That way, all configure scripts will share a cache. Regards, Martin From twisted@itamarst.org Sun Apr 14 16:13:14 2002 From: twisted@itamarst.org (Itamar Shtull-Trauring) Date: Sun, 14 Apr 2002 18:13:14 +0300 Subject: [Python-Dev] Zipfile and tarfile Message-ID: <3CB99C8A.9040804@itamarst.org> Gustavo Niemeyer wrote: > I'd rather choose a default interface similar where possible to that > of zipfile. This makes it easy to include support to tar.gz in programs > already supporting zipfile, and also creates some standardization. I'm > including python-dev in the discussion to know what they think about > this Well, there's a problem with this - zipfile currently gives no way for users to access a file in the zip, except as a string. So, if you've got a 20MB file in your ZIP, and you want to read it in Python, you end up with a 20MB string object. I therefore think that as far as reading ZIP files, Python is very very broken, so limiting tarfile to that API isn't a great idea. I submitted a patch that solved this, which was rejected since I didn't write enough test code or documentation: https://sourceforge.net/tracker/index.php?func=detail&aid=424856&group_id=5470&atid=305470 So, tarfile should use this updated API for compatability - one day I'll find the time to write those tests and docs. Meanwhile a version of zipfile.py with this feature can be found at http://itamarst.org/software/zipfile.py From tim.one@comcast.net Sun Apr 14 18:27:40 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 14 Apr 2002 13:27:40 -0400 Subject: [Python-Dev] Building on Windows (was Re: A "new" kind of leak) In-Reply-To: Message-ID: [Andy Robinson] > You do know how to turn on command line completion? ;-) Not on Win98, no . There I do more work in a Cygwin shell. > HKCU | HKLM -> Software -> Microsoft -> Command Processor -> > CompletionChar -> 0x00009 > and then the tab key completes. (You may need to restart any > current DOS boxes) > > This works on Win2k and NT4. There's another completion char available at least under Win2K, like the above except PathCompletionChar. The latter only expands to directory names, and regardless of context (CompletionChar sticks to directory names when it knows one is required, like after "cd" or "rd"). > There are better shells available for Win98 somewhere, but the NT/2000 > batch language is pretty capable. Not as well understood, but you can > automate lots of processes. The little bit of .bat automation checked into Python's CVS is written to the lowest common Windows denominator (command.com). The automation I've written beyond that is unique to me (which isn't good), and relies on batch run-capture-parse facilities supplied by my editor (which is worse, since I'm its only known user ). > Not that I dispute yur basic thesis - we've managed to avoid > branching at ReportLab to date for exactly this reason :-) So do older releases never get bugfixes? Or you've got no bugs? In either case, can you keep that up for 10 years ? Sell Python support on the side, and if there's an actual paying market for it, it's OK by us if you get filthy rich off it. From pedroni@inf.ethz.ch Sun Apr 14 18:53:16 2002 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Sun, 14 Apr 2002 19:53:16 +0200 Subject: [Python-Dev] devfaq on python.org broken link Message-ID: <001101c1e3dd$41b8eb40$6d94fea9@newmexico> The link in "4.1. How to install cvs and ssh" http://www.python.org/dev/devfaq.html#w1 is broken, http://python.sourceforge.net/winssh.txt needs to be copied to http://www.python.org/dev/winssh.txt regards. From martin@v.loewis.de Sun Apr 14 19:05:45 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 14 Apr 2002 20:05:45 +0200 Subject: [Python-Dev] Zipfile and tarfile In-Reply-To: <3CB99C8A.9040804@itamarst.org> References: <3CB99C8A.9040804@itamarst.org> Message-ID: Itamar Shtull-Trauring writes: > I submitted a patch that solved this, which was rejected since I > didn't write enough test code or documentation: Another such patch was submitted as http://python.org/sf/503592. I personally believe that the method on zipfiles to create a file-like object for archive members should be called .open(). Regards, Martin From twisted@itamarst.org Sun Apr 14 20:13:43 2002 From: twisted@itamarst.org (Itamar Shtull-Trauring) Date: Sun, 14 Apr 2002 21:13:43 +0200 Subject: [Python-Dev] Zipfile and tarfile References: <3CB99C8A.9040804@itamarst.org> Message-ID: <3CB9D4E7.3030501@itamarst.org> Martin v. Loewis wrote: > Another such patch was submitted as http://python.org/sf/503592. I > personally believe that the method on zipfiles to create a file-like > object for archive members should be called .open(). This is a patch for writing to disk. My patch gave the programmer a file-like interface, decompressing from the ZIP file on the fly, so this patch's functionality could be implemented with my patch using shutil.copyobj() or whatever it's called. The name is not important, open() works too. From nas@python.ca Sun Apr 14 19:48:37 2002 From: nas@python.ca (Neil Schemenauer) Date: Sun, 14 Apr 2002 11:48:37 -0700 Subject: [Python-Dev] Building on Windows (was Re: A "new" kind of leak) In-Reply-To: ; from tim.one@comcast.net on Sun, Apr 14, 2002 at 01:27:40PM -0400 References: Message-ID: <20020414114837.A21131@glacier.arctrix.com> Tim Peters wrote: > The automation I've written beyond that is unique to me (which isn't > good), and relies on batch run-capture-parse facilities supplied by my > editor (which is worse, since I'm its only known user ). Source Insight? I tried it out yesterday. It looks pretty cool. If they ported it to Linux (Qt?) I would consider spending the $250 for it. Unfortunately it crashes under the latest version of Wine. I'm going to have to look at cscope one of these days. Neil From tim.one@comcast.net Sun Apr 14 20:31:34 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 14 Apr 2002 15:31:34 -0400 Subject: [Python-Dev] Building on Windows (was Re: A "new" kind of leak) In-Reply-To: <20020414114837.A21131@glacier.arctrix.com> Message-ID: [Neil Schemenauer] > Source Insight? Yup, that's it. > I tried it out yesterday. It looks pretty cool. If they ported it to > Linux (Qt?) I would consider spending the $250 for it. It's Win32+Intel only. All evidence to date suggests it's a one-man shop, and I'm pretty sure the founder answered the phone to take my order for 2.0 originally, and again a year later when I bought the 3.0 upgrade. Ordering over the web is a New Feature (and about time, if the founder really was answering the phone!). > Unfortunately it crashes under the latest version of Wine. I have a cure: upgrade to Windows . > I'm going to have to look at cscope one of these days. It sounds similar in building a database of C symbols, but much less dynamic. SI keeps the database up to date by magic, via background threads that reparse your code while you're editing, and notice when, e.g., a CVS update changes any files in the project. Lots of info is also displayed "by magic". For example, if the cursor happens to be on or in the token "ulong" while I'm editing obmalloc.c, the Context Window automatically displays ulong's definition (plus surrounding lines). The Relation Window also automatically displays a navigation tree showing all uses of ulong in the project. I found this very distracting at first, but now I don't want to work without it. For example, if I'm looking at frameobject.h, simply putting the cursor on f_restricted shows me immediately it's used only twice in the project, and gives me a clickable tree to display those uses in the Context Window (all while leaving the main edit window alone). It can get pretty silly, though. For example, you can tell the Relation WIndow (BTW, you can create any number of those) to display functions called by the current function (or functions calling the current function) only if the percentage of its lines containing control-flow operators is between 37% and 89%. I'd rather Ray spent his time making the editor scriptable via Python, or even doing something commercially worthless like a Linux port . would-that-it-were-open-source-ly y'rs - tim From akuchlin@mems-exchange.org Sun Apr 14 20:32:30 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Sun, 14 Apr 2002 15:32:30 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <15543.41275.421891.469392@anthem.wooz.org> References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <15543.30585.922471.967802@12-248-41-177.client.attbi.com> <15543.37452.98720.899406@anthem.wooz.org> <15543.40546.371491.155932@12-248-41-177.client.attbi.com> <15543.41275.421891.469392@anthem.wooz.org> Message-ID: <20020414193230.GA12505@ute.mems-exchange.org> On Fri, Apr 12, 2002 at 11:08:43PM -0400, Barry A. Warsaw wrote: >Yes. Move getopt.py to getopt/_getopt.py and put this in >getopt/__init__.py: Why not just put the code in getopt/__init__.py, or in a submodule of getopt/ ? Tidier that way... --amk From tim.one@comcast.net Sun Apr 14 20:50:31 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 14 Apr 2002 15:50:31 -0400 Subject: [Python-Dev] A "new" kind of leak In-Reply-To: <20020413071937.A13273@glacier.arctrix.com> Message-ID: [Neil Schemenauer] > PyCFunctionObject and PyMethodObject also use free lists and set the GC > flag. I haven't yet been able to provoke unbounded process growth using these, although I confess I haven't dedicated my life to it . The usual outcome is that creating one of these guys also involves creating another kind of gc'ed object (like a class instance) that doesn't use a free list, and that's also involved in the cycle, and then the allocations of that latter kind of object trigger gc regularly. For example, import gc gc.set_debug(gc.DEBUG_STATS) class C: def f(self): self.f = self.f # create a loop while 1: C().f() That creates tons of cycles, but every 1000-or-so times around the loop, the excess of instance objects allocated triggers gc into cleaning them all up. That prevents the bound method object free list from growing more than 1000-or-so entries too. From barry@zope.com Sun Apr 14 23:18:01 2002 From: barry@zope.com (Barry A. Warsaw) Date: Sun, 14 Apr 2002 18:18:01 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <15543.30585.922471.967802@12-248-41-177.client.attbi.com> <15543.37452.98720.899406@anthem.wooz.org> <200204131047.g3DAlD520681@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15546.25.69774.597342@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> To the contrary. Only problem is how to prevent loading all GvR> of optik when one imports getopt.getopt (and getopt.error). Too bad modules can't have __getattr__'s :) Seriously, I wonder if it's time to write (or adopt) a general autoloading facility for Python? -Barry From barry@zope.com Sun Apr 14 23:18:42 2002 From: barry@zope.com (Barry A. Warsaw) Date: Sun, 14 Apr 2002 18:18:42 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <15543.30585.922471.967802@12-248-41-177.client.attbi.com> <15543.37452.98720.899406@anthem.wooz.org> <15543.40546.371491.155932@12-248-41-177.client.attbi.com> <15543.41275.421891.469392@anthem.wooz.org> <20020414193230.GA12505@ute.mems-exchange.org> Message-ID: <15546.66.341643.506779@anthem.wooz.org> >>>>> "AK" == Andrew Kuchling writes: >> Yes. Move getopt.py to getopt/_getopt.py and put this in >> getopt/__init__.py: AK> Why not just put the code in getopt/__init__.py, or in a AK> submodule of getopt/ ? Tidier that way... You could do the former, but if you did the latter, you'd still need to map those names to the getopt package, for backwards compatibility. -Barry From pedroni@inf.ethz.ch Sun Apr 14 23:40:43 2002 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Mon, 15 Apr 2002 00:40:43 +0200 Subject: [Python-Dev] Adding Optik to 2.3 standard library References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net><15543.30585.922471.967802@12-248-41-177.client.attbi.com><15543.37452.98720.899406@anthem.wooz.org><200204131047.g3DAlD520681@pcp742651pcs.reston01.va.comcast.net> <15546.25.69774.597342@anthem.wooz.org> Message-ID: <019601c1e405$69e3f100$6d94fea9@newmexico> From: Barry A. Warsaw > > >>>>> "GvR" == Guido van Rossum writes: > > GvR> To the contrary. Only problem is how to prevent loading all > GvR> of optik when one imports getopt.getopt (and getopt.error). > > Too bad modules can't have __getattr__'s :) > > Seriously, I wonder if it's time to write (or adopt) a general > autoloading facility for Python? > Seriously, what do you mean by that? I imagine ugly things . Btw, Jython (as you might now) has deep-auto-importing, that means import pkg1.pkg2 pkg1.pkg2.f() in Jython can be written as: import pkg1 pkg1.pkg2.f() (yes you should still import the first-level package), this was intended for Java imports but then extended to python packages too for symmetry. OTOH is a great way to write non-portable (to CPython) jython code . regards. From barry@zope.com Mon Apr 15 00:30:58 2002 From: barry@zope.com (Barry A. Warsaw) Date: Sun, 14 Apr 2002 19:30:58 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <15543.30585.922471.967802@12-248-41-177.client.attbi.com> <15543.37452.98720.899406@anthem.wooz.org> <200204131047.g3DAlD520681@pcp742651pcs.reston01.va.comcast.net> <15546.25.69774.597342@anthem.wooz.org> <019601c1e405$69e3f100$6d94fea9@newmexico> Message-ID: <15546.4402.952111.415453@anthem.wooz.org> >>>>> "SP" == Samuele Pedroni writes: SP> Seriously, what do you mean by that? I imagine ugly things SP> . Naw, I have a beautiful mind. :) SP> Btw, Jython SP> (as you might now) has deep-auto-importing, that means SP> import pkg1.pkg2 SP> pkg1.pkg2.f() SP> in Jython can be written as: SP> import pkg1 SP> pkg1.pkg2.f() Indeed, and that's exactly what I'd like to see, although I don't mind it if packages have to be specially instrumented to provide this behavior. SP> (yes you should still import the first-level package), SP> this was intended for Java imports but then SP> extended to python packages too for symmetry. SP> OTOH is a great way to write non-portable SP> (to CPython) jython code . Another good reason to add something like this! jython-is-a-shining-beacon-ly y'rs, -Barry From vinay_sajip@red-dove.com Mon Apr 15 03:04:39 2002 From: vinay_sajip@red-dove.com (Vinay Sajip) Date: Mon, 15 Apr 2002 03:04:39 +0100 Subject: [Python-Dev] PEP 282 and Python Logging: What Next? Message-ID: <000701c1e421$edf38de0$652b6992@alpha> Almost exactly a month ago I released a logging module which, by dint of drawing from the same influences (log4j, java.util.logging), was fairly closely aligned with PEP 282 (written by Trent Mick). And in that month, suggestions, patches and bug reports have been gratefully received from the community, which have helped to remove some of the rough edges from the module. In PEP 283 (Python 2.3 Release Schedule), PEP 282 is mentioned as one of the planned features for 2.3. It also gets a thumbs up from Guido in the Parade of the PEPs, in that he expresses a hope that PEP 282 can be implemented in 2.3. Clearly, as author of a module which pretty much does everything that 282 calls for (and then some), I'd like the module to be accepted into the standard library and become part of 2.3. I do hope I'm not being too presumptuous in suggesting this, and putting forward my logging module as a reference implementation for PEP 282. Assuming that I'm not ... Being new to the Python enhancement process, I read PEPs 1 and 2. The section in PEP 2 entitled "Acceptance Procedure" talks about the need for a library PEP and a reference implementation. PEP 1 says that the PEP should be accepted before work is started on a reference implementation; however, in this case, since the PEP and the implementation were independently implemented, the latter already exists and in some areas covers ground which is not addressed in detail in the PEP. It's also true that while Trent Mick is the author/champion of the PEP, I am the author/champion of the reference implementation. Comments that were made on the PEP on Python-dev have been generally favourable, the main negatives focusing on lack of detail in one or two areas (e.g. configuration) and the absence of concrete APIs. IMO the concrete API question should be able to be settled now that an implementation exists and is (apparently) proving useful for quite a few people. There are also minor variations (e.g in naming) between implementation and PEP, which need reconciling. The PEP can also be fleshed out in areas (such as configuration) which were not detailed enough for some reviewers. The PEP hasn't been updated since February, and is still in draft status. Guessing that Trent is probably busy on other stuff, on a couple of occasions I've asked for his plans and offered him assistance in updating the PEP and getting it ready for submission, but for whatever reason I've not had any responses. So I'm wondering - what happens next, in terms of PEP acceptance and integrating the module into 2.3? I'd certainly like to move things along and have time available to make things happen. The status of the module is that the implementation is largely feature complete (to my biased view), and has had a moderate amount of testing, both by me and by other users, with no significant fallout having been reported. Documentation is largely in docstrings (practically everything has a docstring - about a third of the module's length is docstrings) and I'll be happy to work it up into LaTeX for inclusion with the standard library docs. There is some work on packaging still to do (it's still in one file and roughly the same size as pydoc; I've not had any feedback as to whether core and handlers should be separated). In terms of testing - there is a bevy of test scripts for testing particular features of the module, which are part of the current distribution at http://www.red-dove.com/python_logging.html Please advise on what you think should happen next. Regards Vinay Sajip From greg@cosc.canterbury.ac.nz Mon Apr 15 03:04:57 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Mon, 15 Apr 2002 14:04:57 +1200 (NZST) Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <15543.5316.761699.16705@anthem.wooz.org> Message-ID: <200204150204.OAA01234@s454.cosc.canterbury.ac.nz> barry@zope.com (Barry A. Warsaw): > Also, is "optik" the right name for the package in the standard > library? "getopt" is taken, but how about "options"? Maybe "arguments" would be better. As someone has pointed out before, not all so-called options are optional! Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Mon Apr 15 03:24:29 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Mon, 15 Apr 2002 14:24:29 +1200 (NZST) Subject: [Python-Dev] A "new" kind of leak In-Reply-To: Message-ID: <200204150224.OAA01238@s454.cosc.canterbury.ac.nz> Tim Peters : > What is surprising is that it also leaks *memory* in 2.2 and current CVS, > although it doesn't leak *objects* anymore, > An effective solution would be to bound the size of the frameobject free > list: I don't think that's the right solution. Won't the same problem occur with all the other kinds of free list as well? You'd have to put bounds on all of them. Seems to me the correct solution is to count allocs/frees from/to any free list along with memory allocs/frees for the purpose of deciding when to do a gc. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From mjm42@comcast.net Mon Apr 15 04:21:06 2002 From: mjm42@comcast.net (Michael McLay) Date: Sun, 14 Apr 2002 23:21:06 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <15546.25.69774.597342@anthem.wooz.org> References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <200204131047.g3DAlD520681@pcp742651pcs.reston01.va.comcast.net> <15546.25.69774.597342@anthem.wooz.org> Message-ID: <0GUL00FKCAML1F@mtaout45-02.icomcast.net> On Sunday 14 April 2002 06:18 pm, Barry A. Warsaw wrote: > >>>>> "GvR" == Guido van Rossum writes: > > GvR> To the contrary. Only problem is how to prevent loading all > GvR> of optik when one imports getopt.getopt (and getopt.error). > > Too bad modules can't have __getattr__'s :) > > Seriously, I wonder if it's time to write (or adopt) a general > autoloading facility for Python? Pmw (Python Megawidgets, http://pmw.sourceforge.net/) has a lazy loader. It is loaded by __init__.py when the Pmw package is loaded. The loader is in Pmw/Pmw_0_8_5/lib/PmwLoader.py From tim.one@comcast.net Mon Apr 15 05:07:33 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 15 Apr 2002 00:07:33 -0400 Subject: [Python-Dev] A "new" kind of leak In-Reply-To: <200204150224.OAA01238@s454.cosc.canterbury.ac.nz> Message-ID: >> An effective solution would be to bound the size of the frameobject >> free list: [Greg Ewing] > I don't think that's the right solution. Won't the same problem > occur with all the other kinds of free list as well? You'd have > to put bounds on all of them. tuple free lists were already bounded. Neil found what look to be the only other two possibilities in the core, but nobody has been able to provoke an actual leak out of them (this has already been gone over here, and I don't want to repeat it). > Seems to me the correct solution is to count allocs/frees from/to > any free list along with memory allocs/frees for the purpose of > deciding when to do a gc. This possibility was mentioned in the bug report, but would cost an extra call per allocation (please don't argue about that unless you're familiar with Python's gc code -- it's a tiny bug, and we're not going to do a massive rework). Since all possibilities require that vulnerable types do something they weren't doing before, right-vs-wrong doesn't appear a sensible axis along which to measure; bounding the free list is an easy change to a vulnerable type's dealloc routine. From tim.one@comcast.net Mon Apr 15 05:33:16 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 15 Apr 2002 00:33:16 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects complexobject.c,2.57,2.58 Message-ID: """ Update of /cvsroot/python/python/dist/src/Objects In directory usw-pr-cvs1:/tmp/cvs-serv7877 Modified Files: complexobject.c Log Message: SF bug #543387. Complex numbers implement divmod() and //, neither of which makes one lick of sense. Unfortunately this is documented, so I'm adding a deprecation warning now, so we can delete this silliness, oh, around 2005 or so. Bugfix candidate (At least for 2.2.2, I think.) """ Does it make sense to deprecate divmod() and // for complex numbers while leaving % intact? x % y == x - y * math.floor((x/y).real) makes exactly as much sense to me (approximately none) as x // y == math.floor((x/y).real) made. Python's definitions of % and // make great sense for integral types, but they start to fall apart for floats, and complexes are right out. From aleax@aleax.it Mon Apr 15 08:36:43 2002 From: aleax@aleax.it (Alex Martelli) Date: Mon, 15 Apr 2002 09:36:43 +0200 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <15546.25.69774.597342@anthem.wooz.org> References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <200204131047.g3DAlD520681@pcp742651pcs.reston01.va.comcast.net> <15546.25.69774.597342@anthem.wooz.org> Message-ID: On Monday 15 April 2002 12:18 am, you wrote: > >>>>> "GvR" == Guido van Rossum writes: > > GvR> To the contrary. Only problem is how to prevent loading all > GvR> of optik when one imports getopt.getopt (and getopt.error). > > Too bad modules can't have __getattr__'s :) Wouldn't the same instance-as-module trick as in, e.g. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65207 let you make a "module-with-__getattr__" to most intents&purposes? Alex From magnus@hetland.org Mon Apr 15 10:41:05 2002 From: magnus@hetland.org (Magnus Lie Hetland) Date: Mon, 15 Apr 2002 11:41:05 +0200 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <15546.25.69774.597342@anthem.wooz.org>; from barry@zope.com on Sun, Apr 14, 2002 at 06:18:01PM -0400 References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <15543.30585.922471.967802@12-248-41-177.client.attbi.com> <15543.37452.98720.899406@anthem.wooz.org> <200204131047.g3DAlD520681@pcp742651pcs.reston01.va.comcast.net> <15546.25.69774.597342@anthem.wooz.org> Message-ID: <20020415114105.A18812@idi.ntnu.no> Barry A. Warsaw : > > > >>>>> "GvR" == Guido van Rossum writes: > > GvR> To the contrary. Only problem is how to prevent loading all > GvR> of optik when one imports getopt.getopt (and getopt.error). > > Too bad modules can't have __getattr__'s :) Well, you could always replace the module with an object that can... :) [snip] > > -Barry -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.org From mwh@python.net Mon Apr 15 11:30:51 2002 From: mwh@python.net (Michael Hudson) Date: 15 Apr 2002 11:30:51 +0100 Subject: [Python-Dev] A "new" kind of leak In-Reply-To: Tim Peters's message of "Sat, 13 Apr 2002 14:10:08 -0400" References: Message-ID: <2mu1qdmexw.fsf@starship.python.net> Tim Peters writes: > [Michael Hudson] > > For patches that > > > > cvs up -j blah -j blat file > > > > can handle, I have a setup that make porting them the work of seconds. > > It takes a little while to set up, so I batch them. > > Maybe waiting for a change to show up in the trunk is a better way to go. > Since I was making the trunk change "live", and wasn't going to check > anything in before everything worked on both trunk and branch, -j was > impotent (in the way I happened to do this). Regardless, it won't work for > *this* patch if it's desired in 2.1 (too much has changed). Yes, this particular patch was exceptional in several ways (eg. no chance of applying cleanly to 21-maint, being hard to test automatically, and affecting a particularly critical bit of code). I doubt many of that list apply to the majority of bugfixes (well, quite a few probably don't apply to 21-maint by now). Longer ramblings will appear in another thread in a moment. Cheers, M. -- If you don't have friends with whom to share links and conversation, you have social problems and you should confront them instead of joining a cultlike pseudo-community. -- "Quit Slashdot.org Today!" (http://www.cs.washington.edu/homes/klee/misc/slashdot.html#faq) From mwh@python.net Mon Apr 15 11:39:16 2002 From: mwh@python.net (Michael Hudson) Date: 15 Apr 2002 11:39:16 +0100 Subject: [Python-Dev] Building on Windows (was Re: A "new" kind of leak) In-Reply-To: Neil Schemenauer's message of "Sun, 14 Apr 2002 11:48:37 -0700" References: <20020414114837.A21131@glacier.arctrix.com> Message-ID: <2mr8lhmejv.fsf@starship.python.net> Neil Schemenauer writes: > Tim Peters wrote: > > The automation I've written beyond that is unique to me (which isn't > > good), and relies on batch run-capture-parse facilities supplied by my > > editor (which is worse, since I'm its only known user ). > > Source Insight? I tried it out yesterday. It looks pretty cool. If > they ported it to Linux (Qt?) I would consider spending the $250 for it. > Unfortunately it crashes under the latest version of Wine. I'm going to > have to look at cscope one of these days. cscope's occasionally handy. I find it unwieldy when you have source files spread among several directories (though this may be user stupidity), and TBH I know the bits of the Python source I regularly beat on well enough that I can usually find definitions as fast by hand/grep as I can with cscope. It's more useful when in unfamiliar territory. Cheers, M. -- The "of course, while I have no problem with this at all, it's surely too much for a lesser being" flavor of argument always rings hollow to me. -- Tim Peters, 29 Apr 1998 From niemeyer@conectiva.com Mon Apr 15 12:21:12 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Mon, 15 Apr 2002 08:21:12 -0300 Subject: [Python-Dev] Building on Windows (was Re: A "new" kind of leak) In-Reply-To: <2mr8lhmejv.fsf@starship.python.net> References: <20020414114837.A21131@glacier.arctrix.com> <2mr8lhmejv.fsf@starship.python.net> Message-ID: <20020415082112.A1187@ibook.distro.conectiva> > cscope's occasionally handy. I find it unwieldy when you have source > files spread among several directories (though this may be user I've created a vim plugin that looks for cscope.out in upper directories until it gets into the root directory. This way, you may just create the database recursively (cscope -b -R) in the root of the project, and everything will work fine. > stupidity), and TBH I know the bits of the Python source I regularly > beat on well enough that I can usually find definitions as fast by > hand/grep as I can with cscope. It's more useful when in unfamiliar > territory. Agreed. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From mwh@python.net Mon Apr 15 13:22:26 2002 From: mwh@python.net (Michael Hudson) Date: 15 Apr 2002 13:22:26 +0100 Subject: [Python-Dev] Porting bug fixes (was A "new" kind of leak) In-Reply-To: Tim Peters's message of "Sat, 13 Apr 2002 16:49:18 -0400" References: Message-ID: <2mlmbpm9rx.fsf@starship.python.net> Tim Peters writes: > [Michael Hudson] > > ... > > So what I'm suggesting is that if you want a checkin to be ported to > > release22-maint you should add a specific bit of text to the chickin > > message. Does this seem reasonable? > > Yes, but you have to Pronounce on which specific bit of text you want to > see. I realise this. It will probably be something along the lines of AutoPorter: 21, 22 (assuming you intend the fix to be applied to release21-maint and release22-maint). This obviously needs to be something everyone knows about, so I'll document it under www.python.org/dev in large friendly letters. It's also likely that this will result in an email back saying "do it yourself" if up -j can't do it (i.e. returns non-zero exit status). > It's going to get much more complicated if we intend to backport fixes > across 2 or 3 years of older releases. I predict that's not going to work > unless we establish an easy-to-update patch database recording which patches > have and haven't been applied to each old release, which should and > shouldn't be applied to each old release, and everyone is serious about > keeping that up to date. I've had similar thoughts. Maybe it's time to learn about CGI... > I'm not aware of any commerical organizations with full-time QA > departments that sign up for something so messy, and I'm not > sanguine about our prospects of pulling it off (the older the code, > the more likely "a bugfix" is to create at least as many problems as > it solves; and the more active branches, the more likely fixes to > get dropped on the floor). Indeed. > > Another random point: it would be nice if on checking a bugfix into > > HEAD people *said* if *they* planned to port the fix themselves. > > Otherwise I see the message that says "bugfix candidate", hit they key > > that copies it into my special bugfixes folder, then read on and find > > it's already been ported and have to go and find the copy and delete > > it. TIA. > > I already do all that stuff, so stop yelling at me . Well, I can't remember who has done this to me, but people have. > [Martin v. Loewis] > > If I'm going to commit the same patch onto the maintainance branch, I > > usually don't mark it as "bugfix candidate". If I can be awkward (and in fact, even if I can't ), I'd like to ask for more than that; sometimes people forget to mark bugfix candidates. So I'd like "I am going to check this in to the 22 branch". At least until I get more automation (at which point if you forget to mark your checkin, tough titties). Cheers, M. -- Screaming 14-year-old boys attempting to prove to each other that they are more 3133t than j00. -- Reason #8 for quitting slashdot today, from http://www.cs.washington.edu/homes/klee/misc/slashdot.html From guido@python.org Mon Apr 15 13:44:32 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 15 Apr 2002 08:44:32 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects complexobject.c,2.57,2.58 In-Reply-To: Your message of "Mon, 15 Apr 2002 00:33:16 EDT." References: Message-ID: <200204151244.g3FCiW116681@pcp742651pcs.reston01.va.comcast.net> [Tim] > Does it make sense to deprecate divmod() and // for complex numbers while > leaving % intact? It doesn't, and I've repaired this. However, I'm wondering what to do after we unify all numeric types (*if* we ever decide to do that, which isn't clear). At that point, should these operators be allowed as long as the imaginary part is zero? --Guido van Rossum (home page: http://www.python.org/~guido/) From neal@metaslash.com Mon Apr 15 13:50:24 2002 From: neal@metaslash.com (Neal Norwitz) Date: Mon, 15 Apr 2002 08:50:24 -0400 Subject: [Python-Dev] Deprecating divmod() // % for complex References: <200204151244.g3FCiW116681@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CBACC90.B7FACD76@metaslash.com> Guido van Rossum wrote: > > [Tim] > > Does it make sense to deprecate divmod() and // for complex numbers while > > leaving % intact? > > It doesn't, and I've repaired this. Does it make sense to expand PEP 4 or make a new PEP to point out all these other features which are also deprecated? This would include classes/methods/functions, etc. Neal From guido@python.org Mon Apr 15 13:57:23 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 15 Apr 2002 08:57:23 -0400 Subject: [Python-Dev] Building on Windows (was Re: A "new" kind of leak) In-Reply-To: Your message of "15 Apr 2002 11:39:16 BST." <2mr8lhmejv.fsf@starship.python.net> References: <20020414114837.A21131@glacier.arctrix.com> <2mr8lhmejv.fsf@starship.python.net> Message-ID: <200204151257.g3FCvN217107@pcp742651pcs.reston01.va.comcast.net> > TBH I know the bits of the Python source I regularly > beat on well enough that I can usually find definitions as fast by > hand/grep as I can with cscope. Don't people use TAGS or tags any more? Works for me (in Emacs). Just say "make TAGS" every once in a while. --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Mon Apr 15 14:02:45 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 15 Apr 2002 15:02:45 +0200 Subject: [Python-Dev] Deprecating divmod() // % for complex References: <200204151244.g3FCiW116681@pcp742651pcs.reston01.va.comcast.net> <3CBACC90.B7FACD76@metaslash.com> Message-ID: <3CBACF75.A89C6F44@lemburg.com> Neal Norwitz wrote: > > Guido van Rossum wrote: > > > > [Tim] > > > Does it make sense to deprecate divmod() and // for complex numbers while > > > leaving % intact? > > > > It doesn't, and I've repaired this. > > Does it make sense to expand PEP 4 or make a new PEP to > point out all these other features which are also deprecated? IMHO, it does. > This would include classes/methods/functions, etc. Plus, a section which covers the C API. All entries should include the reasoning and explain ways to upgrade old code or what to do in order to maintain backward compatibility to run old code without change (e.g. to install a backward compatibility add-on distutils package). -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From guido@python.org Mon Apr 15 14:10:54 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 15 Apr 2002 09:10:54 -0400 Subject: [Python-Dev] Re: Deprecating divmod() // % for complex In-Reply-To: Your message of "Mon, 15 Apr 2002 08:50:24 EDT." <3CBACC90.B7FACD76@metaslash.com> References: <200204151244.g3FCiW116681@pcp742651pcs.reston01.va.comcast.net> <3CBACC90.B7FACD76@metaslash.com> Message-ID: <200204151310.g3FDAsF17235@pcp742651pcs.reston01.va.comcast.net> > Guido van Rossum wrote: > > > > [Tim] > > > Does it make sense to deprecate divmod() and // for complex numbers while > > > leaving % intact? > > > > It doesn't, and I've repaired this. > > Does it make sense to expand PEP 4 or make a new PEP to > point out all these other features which are also deprecated? > This would include classes/methods/functions, etc. > > Neal Unclear what the value is. Deprecations are systematically listed in the documentation AFAIK. --Guido van Rossum (home page: http://www.python.org/~guido/) From neal@metaslash.com Mon Apr 15 14:16:42 2002 From: neal@metaslash.com (Neal Norwitz) Date: Mon, 15 Apr 2002 09:16:42 -0400 Subject: [Python-Dev] Re: Deprecating divmod() // % for complex References: <200204151244.g3FCiW116681@pcp742651pcs.reston01.va.comcast.net> <3CBACC90.B7FACD76@metaslash.com> <200204151310.g3FDAsF17235@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CBAD2BA.FA4B695C@metaslash.com> Guido van Rossum wrote: > > > Guido van Rossum wrote: > > > > > > [Tim] > > > > Does it make sense to deprecate divmod() and // for complex numbers while > > > > leaving % intact? > > > > > > It doesn't, and I've repaired this. > > > > Does it make sense to expand PEP 4 or make a new PEP to > > point out all these other features which are also deprecated? > > This would include classes/methods/functions, etc. > > Unclear what the value is. Deprecations are systematically listed in > the documentation AFAIK. So the APIs and doc can be removed. Here's the current list, found by a lot of grepping in the Python modules: htmllib.HTMLParser.do_nextid() pty.master_open() pty.slave_open() pstats.Stats.ignore() rfc822.AddrlistClass string.zfill() Cookie.Cookie() # parameter usage change There are probably others in the C modules. Neal From barry@zope.com Mon Apr 15 14:19:39 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 15 Apr 2002 09:19:39 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <200204131047.g3DAlD520681@pcp742651pcs.reston01.va.comcast.net> <15546.25.69774.597342@anthem.wooz.org> <200204150736.g3F7aq305679@smtp.zope.com> Message-ID: <15546.54123.302300.885560@anthem.wooz.org> >>>>> "AM" == Alex Martelli writes: AM> Wouldn't the same instance-as-module trick as in, e.g. AM> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65207 AM> let you make a "module-with-__getattr__" to most AM> intents&purposes? Yes, nice. Okay, I'll work something up and submit it as a patch to the Optik project on SF. Then Greg can fiddle with it and check it in if he likes it. >>>>> "MM" == Michael McLay writes: MM> Pmw (Python Megawidgets, http://pmw.sourceforge.net/) has a MM> lazy loader. It is loaded by __init__.py when the Pmw package MM> is loaded. The loader is in Pmw/Pmw_0_8_5/lib/PmwLoader.py Seems quite a bit more complicated than the recipe. -Barry From mal@lemburg.com Mon Apr 15 14:26:28 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 15 Apr 2002 15:26:28 +0200 Subject: [Python-Dev] Re: Deprecating divmod() // % for complex References: <200204151244.g3FCiW116681@pcp742651pcs.reston01.va.comcast.net> <3CBACC90.B7FACD76@metaslash.com> <200204151310.g3FDAsF17235@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CBAD504.5CC3B7DF@lemburg.com> Guido van Rossum wrote: > > > Guido van Rossum wrote: > > > > > > [Tim] > > > > Does it make sense to deprecate divmod() and // for complex numbers while > > > > leaving % intact? > > > > > > It doesn't, and I've repaired this. > > > > Does it make sense to expand PEP 4 or make a new PEP to > > point out all these other features which are also deprecated? > > This would include classes/methods/functions, etc. > > > > Neal > > Unclear what the value is. Deprecations are systematically listed in > the documentation AFAIK. On one page ? Perhaps we should automatically add the PEPs to the standard Python documentation as well ?! Some duplication of this information is necessary to increase visibility, IMHO, esp. for those of us who don't regularly read the docs. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From barry@zope.com Mon Apr 15 14:50:53 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 15 Apr 2002 09:50:53 -0400 Subject: [Python-Dev] Building on Windows (was Re: A "new" kind of leak) References: <20020414114837.A21131@glacier.arctrix.com> <2mr8lhmejv.fsf@starship.python.net> <200204151257.g3FCvN217107@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15546.55997.468088.96563@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> Don't people use TAGS or tags any more? Works for me (in GvR> Emacs). Just say "make TAGS" every once in a while. Yes, works great! From barry@zope.com Mon Apr 15 15:12:42 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 15 Apr 2002 10:12:42 -0400 Subject: Autoloader (was Re: [Python-Dev] Adding Optik to 2.3 standard library) References: <200204121701.g3CH17211220@pcp742651pcs.reston01.va.comcast.net> <200204131047.g3DAlD520681@pcp742651pcs.reston01.va.comcast.net> <15546.25.69774.597342@anthem.wooz.org> <200204150736.g3F7aq305679@smtp.zope.com> Message-ID: <15546.57306.511647.294259@anthem.wooz.org> I've now uploaded a patch (really a tarball) to http://sourceforge.net/tracker/index.php?func=detail&aid=544175&group_id=38019&atid=421099 Below is the __init__.py. I found that _copysyms() was necessary because if you just pulled in the exported symbols from say, getopt, getopt.getopt() wouldn't find the helper functions which are global to its namespace. Oh well. It does mean you have to watch out for name collisions when stitching stuff together like this. You also have to play funny games with imports to get them to work right (you can't do intra-package imports through fully-qualified getopt.* imports). -Barry P.S. I think that ImportError probably ought to be an AttributeError -------------------- snip snip -------------------- # Copyright (c) 2002 Python Software Foundation # As of Python 2.3, we now have two command line option parsing systems, the # legacy getopt interface and Greg Ward's Optik. The code below provides a # lazy loading system so that only the option parsers requested are actually # loaded from this submodule. # Autoloader class, see # http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65207 import sys class _Autoloader: # legacy getopt module exports these symbols getopt_exports = ['getopt', 'GetoptError', 'error'] # Optik exports these symbols optik_exports = ['OptionParser', 'SUPPRESS_HELP', 'SUPPRESS_USAGE', 'STD_HELP_OPTION', 'OptionValueError'] def __getattr__(self, name): if name in self.getopt_exports: import _getopt._getopt self._copysyms(_getopt._getopt) elif name in self.optik_exports: # Import the whole Optik world import _getopt.option self._copysyms(_getopt.option) import _getopt.option_parser self._copysyms(_getopt.option_parser) import _getopt.errors self._copysyms(_getopt.errors) else: raise ImportError, name return self.__dict__[name] # If we don't copy all the globals from the module, we'll get exceptions # when publically exported names try to access other globals in the same # module. def _copysyms(self, module): for name in dir(module): self.__dict__[name] = getattr(module, name) # So we can do sub-imports later sys.modules['_getopt'] = sys.modules[__name__] sys.modules[__name__] = _Autoloader() From fdrake@acm.org Mon Apr 15 15:51:40 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 15 Apr 2002 10:51:40 -0400 Subject: [Python-Dev] Re: Deprecating divmod() // % for complex In-Reply-To: <200204151310.g3FDAsF17235@pcp742651pcs.reston01.va.comcast.net> References: <200204151244.g3FCiW116681@pcp742651pcs.reston01.va.comcast.net> <3CBACC90.B7FACD76@metaslash.com> <200204151310.g3FDAsF17235@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15546.59644.504199.808337@grendel.zope.com> Guido van Rossum writes: > Unclear what the value is. Deprecations are systematically listed in > the documentation AFAIK. If that were the only issue, what's the point of maintaining a list in the PEP at all? I've heard requests that the deprecations be more easily located as part of the documentation, so that there's an actual list of all deprecations as a distinct section. There's a fair bit of merit in that, I think. At the moment, the PEP is the closest thing we have to that list. Generating a list of deprecations from the markup elsewhere in the documention shouldn't be too hard, and we could include a section in the "What's New" document for new deprecations easily enough. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From guido@python.org Mon Apr 15 16:02:09 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 15 Apr 2002 11:02:09 -0400 Subject: [Python-Dev] Re: Deprecating divmod() // % for complex In-Reply-To: Your message of "Mon, 15 Apr 2002 10:51:40 EDT." <15546.59644.504199.808337@grendel.zope.com> References: <200204151244.g3FCiW116681@pcp742651pcs.reston01.va.comcast.net> <3CBACC90.B7FACD76@metaslash.com> <200204151310.g3FDAsF17235@pcp742651pcs.reston01.va.comcast.net> <15546.59644.504199.808337@grendel.zope.com> Message-ID: <200204151502.g3FF2AV25900@pcp742651pcs.reston01.va.comcast.net> > Guido van Rossum writes: > > Unclear what the value is. Deprecations are systematically listed in > > the documentation AFAIK. [Fred] > If that were the only issue, what's the point of maintaining a list in > the PEP at all? Beats me. Maybe because the PEP author thought deprecating whole modules was a heavier kind of operation? Maybe because he didn't realize other things could be deprecated as well? Maybe because he planned a separate PEP for those? > I've heard requests that the deprecations be more easily located as > part of the documentation, so that there's an actual list of all > deprecations as a distinct section. There's a fair bit of merit in > that, I think. At the moment, the PEP is the closest thing we have to > that list. Yes, having a summary of deprecation info per release makes sense. Ideally it should be something readily accessible and easy to point to (e.g. from the release webpage). > Generating a list of deprecations from the markup elsewhere in the > documention shouldn't be too hard, and we could include a section in > the "What's New" document for new deprecations easily enough. If that's the way you want to do it. I think there's merit in requiring that some thought goes into the deprecation process. --Guido van Rossum (home page: http://www.python.org/~guido/) From python@rcn.com Mon Apr 15 16:13:13 2002 From: python@rcn.com (Raymond Hettinger) Date: Mon, 15 Apr 2002 11:13:13 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects complexobject.c,2.57,2.58 Message-ID: <002501c1e490$11ac1b80$35f8a4d8@othello> [Tim] > Does it make sense to deprecate divmod() and // for complex numbers while > leaving % intact? [GvR] It doesn't, and I've repaired this. [GvR] However, I'm wondering what to do after we unify all numeric types (*if* we ever decide to do that, which isn't clear). At that point, should these operators be allowed as long as the imaginary part is zero? [RDH] Yes. Ideally, in a unified numeric model, all math operations should accept all numeric types and raise a domain exception (ValueError: math domain error) when an argument falls outside the defined scope just like we do for divide by zero, logs of negative numbers, and inverse trig functions which fall outside a defined domain. .real, .imag, and .conjugate() should be defined for integers and floats as well as complex numbers: (3).imag --> 0 Likewise, divmod() and // should work for complex numbers where the imaginary part is zero. Raymond Hettinger From fdrake@acm.org Mon Apr 15 16:16:50 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 15 Apr 2002 11:16:50 -0400 Subject: [Python-Dev] Re: Deprecating divmod() // % for complex In-Reply-To: <3CBAD2BA.FA4B695C@metaslash.com> References: <200204151244.g3FCiW116681@pcp742651pcs.reston01.va.comcast.net> <3CBACC90.B7FACD76@metaslash.com> <200204151310.g3FDAsF17235@pcp742651pcs.reston01.va.comcast.net> <3CBAD2BA.FA4B695C@metaslash.com> Message-ID: <15546.61154.973803.448931@grendel.zope.com> Neal Norwitz writes: > So the APIs and doc can be removed. Here's the current list, > found by a lot of grepping in the Python modules: The docs cannot be removed when something becomes deprecated, only when it is removed. Deprecated features still need to be documented since someone will need to modify code to not use the deprecated feature. This requires understanding the deprecated material as well as whatever is used to supplant it. It also serves as a place to provide a link to the replacement features. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From paul@svensson.org Mon Apr 15 16:20:28 2002 From: paul@svensson.org (Paul Svensson) Date: Mon, 15 Apr 2002 11:20:28 -0400 (EDT) Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects complexobject.c,2.57,2.58 In-Reply-To: <200204151244.g3FCiW116681@pcp742651pcs.reston01.va.comcast.net> Message-ID: On Mon, 15 Apr 2002, Guido van Rossum wrote: >[Tim] >> Does it make sense to deprecate divmod() and // for complex numbers while >> leaving % intact? > >It doesn't, and I've repaired this. > >However, I'm wondering what to do after we unify all numeric types >(*if* we ever decide to do that, which isn't clear). At that point, >should these operators be allowed as long as the imaginary part is >zero? I wasn't aware that complex numbers were part of the grand unification plan, but of course it makes a lot of sense. It then seems fairly obvious to me that treating a number with zero imaginary part differently than a number with no imaginary part would surprise a lot of people. +1 on allowing these operators to continue to work on complex numbers with a zero imaginary part. /Paul From guido@python.org Mon Apr 15 16:30:13 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 15 Apr 2002 11:30:13 -0400 Subject: [Python-Dev] Optik Message-ID: <200204151530.g3FFUDe26272@pcp742651pcs.reston01.va.comcast.net> I've thought about it some more, and I don't like the autoloader business one bit. I expect it will trip up auxiliary tools, introspection, etc. I don't think that names in the toplevel namespace are such a scarce resource that we can't add a module or package there. How about calling the module OptionParser, since that seems to be the main class exported by Optik? I really don't care if it is a module or a package, as long as this works: from OptionParser import OptionParser op = OptionParser(...suitable args...) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 15 16:32:20 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 15 Apr 2002 11:32:20 -0400 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects complexobject.c,2.57,2.58 In-Reply-To: Your message of "Mon, 15 Apr 2002 11:20:28 EDT." References: Message-ID: <200204151532.g3FFWKG26301@pcp742651pcs.reston01.va.comcast.net> > +1 on allowing these operators to continue to work > on complex numbers with a zero imaginary part. -1 for now. I agree that under the grand numeric unification scheme this must work, but it's not clear that that will ever happen, and until then, there's no point. --Guido van Rossum (home page: http://www.python.org/~guido/) From andy@reportlab.com Mon Apr 15 16:34:10 2002 From: andy@reportlab.com (Andy Robinson) Date: Mon, 15 Apr 2002 16:34:10 +0100 Subject: [Python-Dev] Deprecating divmod() // % for complex In-Reply-To: <20020415132630.30653.99074.Mailman@mail.python.org> Message-ID: > -----Original Message----- > From: python-dev-admin@python.org [mailto:python-dev-admin@python.org]On > Behalf Of python-dev-request@python.org > Sent: 15 April 2002 14:27 > To: python-dev@python.org > Subject: Python-Dev digest, Vol 1 #2152 - 16 msgs > > > Send Python-Dev mailing list submissions to > python-dev@python.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.python.org/mailman/listinfo/python-dev > or, via email, send a message with subject or body 'help' to > python-dev-request@python.org > > You can reach the person managing the list at > python-dev-admin@python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Python-Dev digest..." > > > Today's Topics: > > 1. RE: A "new" kind of leak (Tim Peters) > 2. Re: [Python-checkins] python/dist/src/Objects > complexobject.c,2.57,2.58 (Tim Peters) > 3. Re: Adding Optik to 2.3 standard library (Alex Martelli) > 4. Re: Adding Optik to 2.3 standard library (Magnus Lie Hetland) > 5. Re: A "new" kind of leak (Michael Hudson) > 6. Re: Building on Windows (was Re: A "new" kind of leak) > (Michael Hudson) > 7. Re: Building on Windows (was Re: A "new" kind of leak) > (Gustavo Niemeyer) > 8. Re: Porting bug fixes (was A "new" kind of leak) (Michael Hudson) > 9. Re: [Python-checkins] python/dist/src/Objects > complexobject.c,2.57,2.58 (Guido van Rossum) > 10. Deprecating divmod() // % for complex (Neal Norwitz) > 11. Re: Building on Windows (was Re: A "new" kind of leak) > (Guido van Rossum) > 12. Re: Deprecating divmod() // % for complex (M.-A. Lemburg) > 13. Re: Deprecating divmod() // % for complex (Guido van Rossum) > 14. Re: Deprecating divmod() // % for complex (Neal Norwitz) > 15. Re: Adding Optik to 2.3 standard library (Barry A. Warsaw) > 16. Re: Re: Deprecating divmod() // % for complex (M.-A. Lemburg) > > --__--__-- > > Message: 1 > Date: Mon, 15 Apr 2002 00:07:33 -0400 > From: Tim Peters > Subject: RE: [Python-Dev] A "new" kind of leak > To: python-dev@python.org > > >> An effective solution would be to bound the size of the frameobject > >> free list: > > [Greg Ewing] > > I don't think that's the right solution. Won't the same problem > > occur with all the other kinds of free list as well? You'd have > > to put bounds on all of them. > > tuple free lists were already bounded. Neil found what look to > be the only > other two possibilities in the core, but nobody has been able to > provoke an > actual leak out of them (this has already been gone over here, and I don't > want to repeat it). > > > Seems to me the correct solution is to count allocs/frees from/to > > any free list along with memory allocs/frees for the purpose of > > deciding when to do a gc. > > This possibility was mentioned in the bug report, but would cost an extra > call per allocation (please don't argue about that unless you're familiar > with Python's gc code -- it's a tiny bug, and we're not going to do a > massive rework). > > Since all possibilities require that vulnerable types do something they > weren't doing before, right-vs-wrong doesn't appear a sensible axis along > which to measure; bounding the free list is an easy change to a vulnerable > type's dealloc routine. > > > > > --__--__-- > > Message: 2 > Date: Mon, 15 Apr 2002 00:33:16 -0400 > From: Tim Peters > To: Guido van Rossum > Cc: PythonDev > Subject: [Python-Dev] Re: [Python-checkins] > python/dist/src/Objects complexobject.c,2.57,2.58 > > """ > Update of /cvsroot/python/python/dist/src/Objects > In directory usw-pr-cvs1:/tmp/cvs-serv7877 > > Modified Files: > complexobject.c > Log Message: > SF bug #543387. > > Complex numbers implement divmod() and //, neither of which makes one > lick of sense. Unfortunately this is documented, so I'm adding a > deprecation warning now, so we can delete this silliness, oh, around > 2005 or so. > > Bugfix candidate (At least for 2.2.2, I think.) > """ > > Does it make sense to deprecate divmod() and // for complex numbers while > leaving % intact? > > x % y == x - y * math.floor((x/y).real) > > makes exactly as much sense to me (approximately none) as > > x // y == math.floor((x/y).real) > > made. Python's definitions of % and // make great sense for > integral types, > but they start to fall apart for floats, and complexes are right out. > > > > > --__--__-- > > Message: 3 > From: Alex Martelli > Organization: None in Sight > To: barry@zope.com (Barry A. Warsaw) > Subject: Re: [Python-Dev] Adding Optik to 2.3 standard library > Date: Mon, 15 Apr 2002 09:36:43 +0200 > Cc: python-dev@python.org > > On Monday 15 April 2002 12:18 am, you wrote: > > >>>>> "GvR" == Guido van Rossum writes: > > > > GvR> To the contrary. Only problem is how to prevent loading all > > GvR> of optik when one imports getopt.getopt (and getopt.error). > > > > Too bad modules can't have __getattr__'s :) > > Wouldn't the same instance-as-module trick as in, e.g. > > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65207 > > let you make a "module-with-__getattr__" to most intents&purposes? > > > Alex > > > > --__--__-- > > Message: 4 > Date: Mon, 15 Apr 2002 11:41:05 +0200 > From: Magnus Lie Hetland > To: python-dev@python.org > Subject: Re: [Python-Dev] Adding Optik to 2.3 standard library > > Barry A. Warsaw : > > > > > > >>>>> "GvR" == Guido van Rossum writes: > > > > GvR> To the contrary. Only problem is how to prevent loading all > > GvR> of optik when one imports getopt.getopt (and getopt.error). > > > > Too bad modules can't have __getattr__'s :) > > Well, you could always replace the module with an object that can... > :) > > [snip] > > > > -Barry > > -- > Magnus Lie Hetland The Anygui Project > http://hetland.org http://anygui.org > > > > --__--__-- > > Message: 5 > To: python-dev@python.org > Subject: Re: [Python-Dev] A "new" kind of leak > From: Michael Hudson > Date: 15 Apr 2002 11:30:51 +0100 > > Tim Peters writes: > > > [Michael Hudson] > > > For patches that > > > > > > cvs up -j blah -j blat file > > > > > > can handle, I have a setup that make porting them the work of seconds. > > > It takes a little while to set up, so I batch them. > > > > Maybe waiting for a change to show up in the trunk is a better > way to go. > > Since I was making the trunk change "live", and wasn't going to check > > anything in before everything worked on both trunk and branch, -j was > > impotent (in the way I happened to do this). Regardless, it > won't work for > > *this* patch if it's desired in 2.1 (too much has changed). > > Yes, this particular patch was exceptional in several ways (eg. no > chance of applying cleanly to 21-maint, being hard to test > automatically, and affecting a particularly critical bit of code). > > I doubt many of that list apply to the majority of bugfixes (well, > quite a few probably don't apply to 21-maint by now). > > Longer ramblings will appear in another thread in a moment. > > Cheers, > M. > > -- > If you don't have friends with whom to share links and conversation, > you have social problems and you should confront them instead of > joining a cultlike pseudo-community. -- "Quit Slashdot.org Today!" > (http://www.cs.washington.edu/homes/klee/misc/slashdot.html#faq) > > > > --__--__-- > > Message: 6 > To: python-dev@python.org > Subject: Re: [Python-Dev] Building on Windows (was Re: A "new" > kind of leak) > From: Michael Hudson > Date: 15 Apr 2002 11:39:16 +0100 > > Neil Schemenauer writes: > > > Tim Peters wrote: > > > The automation I've written beyond that is unique to me (which isn't > > > good), and relies on batch run-capture-parse facilities supplied by my > > > editor (which is worse, since I'm its only known user ). > > > > Source Insight? I tried it out yesterday. It looks pretty cool. If > > they ported it to Linux (Qt?) I would consider spending the $250 for it. > > Unfortunately it crashes under the latest version of Wine. I'm going to > > have to look at cscope one of these days. > > cscope's occasionally handy. I find it unwieldy when you have source > files spread among several directories (though this may be user > stupidity), and TBH I know the bits of the Python source I regularly > beat on well enough that I can usually find definitions as fast by > hand/grep as I can with cscope. It's more useful when in unfamiliar > territory. > > Cheers, > M. > > -- > The "of course, while I have no problem with this at all, it's > surely too much for a lesser being" flavor of argument always > rings hollow to me. -- Tim Peters, 29 Apr 1998 > > > > --__--__-- > > Message: 7 > Date: Mon, 15 Apr 2002 08:21:12 -0300 > From: Gustavo Niemeyer > To: Michael Hudson > Cc: python-dev@python.org > Subject: Re: [Python-Dev] Building on Windows (was Re: A "new" > kind of leak) > > > cscope's occasionally handy. I find it unwieldy when you have source > > files spread among several directories (though this may be user > > I've created a vim plugin that looks for cscope.out in upper directories > until it gets into the root directory. This way, you may just create the > database recursively (cscope -b -R) in the root of the project, and > everything will work fine. > > > stupidity), and TBH I know the bits of the Python source I regularly > > beat on well enough that I can usually find definitions as fast by > > hand/grep as I can with cscope. It's more useful when in unfamiliar > > territory. > > Agreed. > > -- > Gustavo Niemeyer > > [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] > > > > --__--__-- > > Message: 8 > To: python-dev@python.org > Subject: Re: [Python-Dev] Porting bug fixes (was A "new" kind of leak) > From: Michael Hudson > Date: 15 Apr 2002 13:22:26 +0100 > > Tim Peters writes: > > > [Michael Hudson] > > > ... > > > So what I'm suggesting is that if you want a checkin to be ported to > > > release22-maint you should add a specific bit of text to the chickin > > > message. Does this seem reasonable? > > > > Yes, but you have to Pronounce on which specific bit of text you want to > > see. > > I realise this. It will probably be something along the lines of > > AutoPorter: 21, 22 > > (assuming you intend the fix to be applied to release21-maint and > release22-maint). This obviously needs to be something everyone knows > about, so I'll document it under www.python.org/dev in large friendly > letters. > > It's also likely that this will result in an email back saying "do it > yourself" if up -j can't do it (i.e. returns non-zero exit status). > > > It's going to get much more complicated if we intend to backport fixes > > across 2 or 3 years of older releases. I predict that's not > going to work > > unless we establish an easy-to-update patch database recording > which patches > > have and haven't been applied to each old release, which should and > > shouldn't be applied to each old release, and everyone is serious about > > keeping that up to date. > > I've had similar thoughts. Maybe it's time to learn about CGI... > > > I'm not aware of any commerical organizations with full-time QA > > departments that sign up for something so messy, and I'm not > > sanguine about our prospects of pulling it off (the older the code, > > the more likely "a bugfix" is to create at least as many problems as > > it solves; and the more active branches, the more likely fixes to > > get dropped on the floor). > > Indeed. > > > > Another random point: it would be nice if on checking a bugfix into > > > HEAD people *said* if *they* planned to port the fix themselves. > > > Otherwise I see the message that says "bugfix candidate", hit they key > > > that copies it into my special bugfixes folder, then read on and find > > > it's already been ported and have to go and find the copy and delete > > > it. TIA. > > > > I already do all that stuff, so stop yelling at me . > > Well, I can't remember who has done this to me, but people have. > > > [Martin v. Loewis] > > > If I'm going to commit the same patch onto the maintainance branch, I > > > usually don't mark it as "bugfix candidate". > > If I can be awkward (and in fact, even if I can't ), I'd like to > ask for more than that; sometimes people forget to mark bugfix > candidates. So I'd like "I am going to check this in to the 22 > branch". At least until I get more automation (at which point if you > forget to mark your checkin, tough titties). > > Cheers, > M. > > -- > Screaming 14-year-old boys attempting to prove to each other that > they are more 3133t than j00. > -- Reason #8 for quitting slashdot today, from > http://www.cs.washington.edu/homes/klee/misc/slashdot.html > > > > --__--__-- > > Message: 9 > To: Tim Peters > cc: PythonDev > From: Guido van Rossum > Date: Mon, 15 Apr 2002 08:44:32 -0400 > Subject: [Python-Dev] Re: [Python-checkins] > python/dist/src/Objects complexobject.c,2.57,2.58 > > [Tim] > > Does it make sense to deprecate divmod() and // for complex > numbers while > > leaving % intact? > > It doesn't, and I've repaired this. > > However, I'm wondering what to do after we unify all numeric types > (*if* we ever decide to do that, which isn't clear). At that point, > should these operators be allowed as long as the imaginary part is > zero? > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > > > --__--__-- > > Message: 10 > Date: Mon, 15 Apr 2002 08:50:24 -0400 > From: Neal Norwitz > Organization: MetaSlash, Inc. > To: Guido van Rossum > CC: Tim Peters , PythonDev > Subject: [Python-Dev] Deprecating divmod() // % for complex > > Guido van Rossum wrote: > > > > [Tim] > > > Does it make sense to deprecate divmod() and // for complex > numbers while > > > leaving % intact? > > > > It doesn't, and I've repaired this. > > Does it make sense to expand PEP 4 or make a new PEP to > point out all these other features which are also deprecated? > This would include classes/methods/functions, etc. > > Neal > > > > --__--__-- > > Message: 11 > To: Michael Hudson > cc: python-dev@python.org > Subject: Re: [Python-Dev] Building on Windows (was Re: A "new" > kind of leak) > From: Guido van Rossum > Date: Mon, 15 Apr 2002 08:57:23 -0400 > > > TBH I know the bits of the Python source I regularly > > beat on well enough that I can usually find definitions as fast by > > hand/grep as I can with cscope. > > Don't people use TAGS or tags any more? Works for me (in Emacs). > Just say "make TAGS" every once in a while. > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > > > --__--__-- > > Message: 12 > Date: Mon, 15 Apr 2002 15:02:45 +0200 > From: "M.-A. Lemburg" > Organization: eGenix.com Software GmbH > To: Neal Norwitz > CC: Guido van Rossum , Tim Peters , > PythonDev > Subject: Re: [Python-Dev] Deprecating divmod() // % for complex > > Neal Norwitz wrote: > > > > Guido van Rossum wrote: > > > > > > [Tim] > > > > Does it make sense to deprecate divmod() and // for complex > numbers while > > > > leaving % intact? > > > > > > It doesn't, and I've repaired this. > > > > Does it make sense to expand PEP 4 or make a new PEP to > > point out all these other features which are also deprecated? > > IMHO, it does. > > > This would include classes/methods/functions, etc. > > Plus, a section which covers the C API. > > All entries should include the reasoning and explain ways to > upgrade old code or what to do in order to maintain backward > compatibility to run old code without change (e.g. to install > a backward compatibility add-on distutils package). > > -- > Marc-Andre Lemburg > CEO eGenix.com Software GmbH > ______________________________________________________________________ > Company & Consulting: http://www.egenix.com/ > Python Software: http://www.egenix.com/files/python/ > > > > --__--__-- > > Message: 13 > To: Neal Norwitz > cc: Tim Peters , PythonDev > From: Guido van Rossum > Date: Mon, 15 Apr 2002 09:10:54 -0400 > Subject: [Python-Dev] Re: Deprecating divmod() // % for complex > > > Guido van Rossum wrote: > > > > > > [Tim] > > > > Does it make sense to deprecate divmod() and // for complex > numbers while > > > > leaving % intact? > > > > > > It doesn't, and I've repaired this. > > > > Does it make sense to expand PEP 4 or make a new PEP to > > point out all these other features which are also deprecated? > > This would include classes/methods/functions, etc. > > > > Neal > > Unclear what the value is. Deprecations are systematically listed in > the documentation AFAIK. > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > > > --__--__-- > > Message: 14 > Date: Mon, 15 Apr 2002 09:16:42 -0400 > From: Neal Norwitz > Organization: MetaSlash, Inc. > To: Guido van Rossum > CC: Tim Peters , PythonDev > Subject: [Python-Dev] Re: Deprecating divmod() // % for complex > > Guido van Rossum wrote: > > > > > Guido van Rossum wrote: > > > > > > > > [Tim] > > > > > Does it make sense to deprecate divmod() and // for > complex numbers while > > > > > leaving % intact? > > > > > > > > It doesn't, and I've repaired this. > > > > > > Does it make sense to expand PEP 4 or make a new PEP to > > > point out all these other features which are also deprecated? > > > This would include classes/methods/functions, etc. > > > > Unclear what the value is. Deprecations are systematically listed in > > the documentation AFAIK. > > So the APIs and doc can be removed. Here's the current list, > found by a lot of grepping in the Python modules: > > htmllib.HTMLParser.do_nextid() > pty.master_open() > pty.slave_open() > pstats.Stats.ignore() > rfc822.AddrlistClass > string.zfill() > > Cookie.Cookie() # parameter usage change > > There are probably others in the C modules. > > Neal > > > > --__--__-- > > Message: 15 > Date: Mon, 15 Apr 2002 09:19:39 -0400 > To: Alex Martelli > Cc: python-dev@python.org > Subject: Re: [Python-Dev] Adding Optik to 2.3 standard library > From: barry@zope.com (Barry A. Warsaw) > > > >>>>> "AM" == Alex Martelli writes: > > AM> Wouldn't the same instance-as-module trick as in, e.g. > > AM> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65207 > > AM> let you make a "module-with-__getattr__" to most > AM> intents&purposes? > > Yes, nice. Okay, I'll work something up and submit it as a patch to > the Optik project on SF. Then Greg can fiddle with it and check it in > if he likes it. > > >>>>> "MM" == Michael McLay writes: > > MM> Pmw (Python Megawidgets, http://pmw.sourceforge.net/) has a > MM> lazy loader. It is loaded by __init__.py when the Pmw package > MM> is loaded. The loader is in Pmw/Pmw_0_8_5/lib/PmwLoader.py > Perhaps we should automatically add the PEPs to the standard > Python documentation as well ?! Some duplication of this > information is necessary to increase visibility, IMHO, esp. > for those of us who don't regularly read the docs. I for one would find this really handy and interesting. It kind of puts PEPs into the "read-about-Python's-history -in-an-armchair" category rather than requiring active lookup. Think of it as one more manual indexed on the main docco page. - Andy Robinson From novaluz@sp.mailbr.com.br Mon Apr 15 16:50:16 2002 From: novaluz@sp.mailbr.com.br (NovaLuz) Date: Mon, 15 Apr 2002 12:50:16 -0300 Subject: [Python-Dev] Concorra a uma Luminaria - Novaluz - GRATÏS!! Message-ID: <1254169-220024115155016610@sp.mailbr.com.br> This is a multi-part message in MIME format. ------=_NextPart_94915C5ABAF209EF376268C8 Content-Type: multipart/alternative; boundary="----=_NextPart_84815C5ABAF209EF376268C8" ------=_NextPart_84815C5ABAF209EF376268C8 Content-type: text/plain; charset=US-ASCII ------=_NextPart_84815C5ABAF209EF376268C8 Content-Type: text/html; charset=US-ASCII Content-Transfer-Encoding: quoted-printable A NovaLuz est=E1 sorteando um equipamento de Ilumina=E7=E3o de Emer= g=EAncia

A NovaLuz est=E1 sorteando uma Lumin=E1ria Normal e Emerg=EAncia!!!

 

Para participar basta acessar o website: http://www=2Enovaluz=2Ecom=2Ebr/, preencher o formul=E1rio do concurso e ind= icar este site aos amigos=2E

 

Sorteio =96 Dia 30 de Abril/2002


 

Lumin=E1ria Nor= mal e Emerg=EAncia =96 Modelo NL 3x9 NE

 

Equipamento com= fun=E7=E3o 2 em 1=2E Possui 3 l=E2mpadas PL de 9 watts, sendo que duas delas proporci= onam a ilumina=E7=E3o normal para o ambiente e a terceira =E9 acionada automati= camente na falta de energia el=E9trica=2E

 

Autonomia para = a luz de emerg=EAncia: 4 horas

 

 = ;

Boa Sorte!!!<= /b>

 

El=E9trica NovaLuz Ltda

http://www=2Eanovaluz=2Ecom=2Eb= r/

Tel: (11) 222 2699 * Fax: (11) 3331 3033

Luz de Emerg=EAncia & Sensores de Presen=E7a

 

 

------=_NextPart_84815C5ABAF209EF376268C8-- ------=_NextPart_94915C5ABAF209EF376268C8 Content-Type: application/octet-stream; name="image001.gif" Content-Description: image001.gif Content-Id: <38100-2200241151539523804@sp.mailbr.com.br> R0lGODlhmACfAPUYACkpKDk5QjlCQkVDM0lKSXFwaAAY3hUt3jRF1k1e1mNrzmt7znuEe3uEyIV7 couHeJWVjaSdlamnmLWyrZ+iyb3Gvca9ts3IusDAwM7MytnX1N7e5+fe2e7p2fDt6/39/MDAwAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAEAACAAIf4IR2lmIEx1YmUALAAA AACYAJ8AxSkpKDk5QjlCQkVDM0lKSXFwaAAY3hUt3jRF1k1e1mNrzmt7znuEe3uEyIV7couHeJWV jaSdlamnmLWyrZ+iyb3Gvca9ts3IusDAwM7MytnX1N7e5+fe2e7p2fDt6/39/MDAwAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAb+QINwSCwaj8ikcslsOp/QqAJEpRwoVGpimu16 v+CweEwum8/oNFUh5IIaCC8lrq7b7/h8nW1w6/+AgYJ3fH4UCQgKE4OMjY54hVl8RoMZGg8THR0c mhcXH6ChHhkTpRMaHhqqGmesj46Ra0h/GRkQAAUTER4fHh4docGhpBAFBRAQF6jDFxISFsvCvSAQ ERMZqauur2mxWQhYWQZ3GRcA5wAEA8gSvZq+79Ly8/TBqRMSDg4PEhccvKmaPYsGqoMnWx22DfJG BcMBNlbCpcnwYQAuAhgxVpvw4Z8wXqB8udPgD9jHX5re+VpZr2VIaxEkdJDAC5vCQAxBRNT5sA7+ qYwZcU14gEpDB4AceoVMyYGTR5fyAEJdykFDVU0aCijj6IHjzT85t2SxUkcDUAICAqzjN0HgKQ00 V8qdO1cpyKlR70LV9OHCKUYMMSw4cCCBRDUTzmJMt7GcLgjO2lqoZUkbJ7pzq6qq+i9zZ7lNMf9D tbIA0QKoIH/Nk/OP2cXn0MoWIEC2bcW4gaJLR4DBIsePJSy6YOECCOIgammzumE5ZWIFJETI9QD1 wjaNlnsAoasA7Nq5caebUACAxQgwH0BQX+1YsQcM2DmLKTyCA2NnYxM4VioCsQgPpNJIa9wEstwH CZUCwX3IxIRMgxIUE50EBPCmGGqqFOgFgRr+dujhgNh9KOKIgnBI4okokmFiiiy2CMKKLsY4Iowy 1qihNwocgMCOPI5j448g9jHWFVk45AeQSOqBIx0UGEBkklAqGSKJtVgQATQZgLBKlAUSiEECBiCA QSCkALCeBgimgkxbJvWiAT7CtfnRm+vZpA2XX9CYBzYVDrCOMwUphdc8o0hQnQPWRJOKBc70o1co HeiSaDZb3jhlFxNQcNhEGZSnH0YRynSUS+/wNeip9sC13gPKGGUVMBStJqWQXURkgBUL+NSBYrVF eEEqGwDEkj2bkHJKL8PWwxQ8KR2VbD0gBKhML8oEeaSRFDTwpBoP5BcAZFdqMBlpIpWL6rn+9sjl Ei8SZHgdrbUe4GSudVwgXlsA8jMQZqBl4yy/AAsbsFSihOQLXNDEVUBC77pxCGENLHJHt7rtBgB4 FaOznzW1+CVdKco9R9lmHnzmi2bLWZbychxMdwFkHWTQbsOMaIBOALNhVBvG4YUnAC4SeAeZfEMp FoDG/CU3AYTC+WVKKRaYEuF690lc4qUnptyLLgxgFEGFQin44IOmVBMhfTFNp54x7kFAXni4UCor IHriSYbW93R79DkRuHti3XYHriLWghfeDeGGJz44vIo3PgbgjhcOeeSB53TIjg1QrjiOR4Lgo+aV X6ppraBLPiU4X3xeOpeWI4JA5quHzrj+TpvGjqc3CcAOwhUNqG47kpP//iOMvtOSpVUlcaBMqKTd eYfft09JgQLUV089mRncx6o7wBhUijKPioLPpB84H3lgYCZgNSDYeAcgKpugCQqd+/BzwahuMioc QanqIpzcrEOcIGrhgQL4aUICO5UHKmCoAjjAbQQRF5zuFxWZuc1O0JsR4XakgARsyycesEg61gEZ g4lCXeiyx8uM8YAIUDAkF0CPC8OHIJlFJyGVspYcDjAmECQgAXbIQAd2k5Gl0QR/vehMumiYQmkQ 4wFQ7IdJUiGd5HFgf7CQnqa2uEWffMBTZ2HHJpgoDM00qy5NdMn9cAgXUExAQIDBGur+fFS8MvAJ N+uxQCrwV7KoNEUVSKwHPJqSkmcNaigfyET5Mkg3Ae7pNbqhTluM1Yta2AMUFwBOS1AIlUGekZOX qMonIjU3SMjxMHUkg8wUI5R+ZMACuXgUGdN4QpFMxRfluOLW4Hi12XHHdbUzQ6fO8rMWNk1mbZEA NjBTMEHRspPwsEo0gNGWUtohLPTiiU+I6bVkRC0yECxZUy7DkoGZiy7IwgsnpXGwB5SPZrI4Qh0S gxt8lIIfxVHOKkxGF9KgLDTmDGjAPrABArTrAUfRijXVwKFDOKkOEGAlbyzWm0xaQkEtlIwFJlMO faqCLpwpmdys4oHmcKA5m9nMBkb+U5TOFKMDD9CjLuIIr+lZr3p18E5QzvGz2+wMLTgzGtIu+EpJ Cec3WUqqRwH6i5GuTBsZ+NrLIFAOCCz0cL7Uw2s0FtTa4OynPfNZOvCz09jw52kwMUU5omac5FTG OUvjx3ocWIAs9bJzf6DQbn4K1rCKpwDTQUZ0yNM2s6FHOhGYK1kvBrb9uG0CUZOUfJBRHQxdFat4 dQTeQDAdsFkoKAQgj5/WYQ18jE2Gia2GfA47HxZW1m0MCKpQ0wGNyzLUkS7SmgYkJFhryKdRaXuQ McZGNfLIVjFv8VDwoqQNai0NAgxgwDHQcxugBKAAvBTRcle3shZtV3gx+i54vYv+2/FCSbzmRRF6 00ui9bJXu+V9r43cK98O0be+XYqCfvfL3/76979OuKmAB0zgAhv4wAhOsIIXzOAGDzi++HXRfSP8 CIZoawGaQgAPKcwib8BhhxxOEedSF2L1apGLXSxxey/1wQ+qWLmnq90BXqxBeIklXjSGL+MunK3C 5FjHmf3xirMq5CH7wYMaVkAPi2wpeKGOCho+gO6YXGHpSXkOM9YJlbkx4S3flshebnKQqZBKPVCk GZ6ISVsACAhW2LbD5S0zOcr3R83sIolwOtZyzNvlMWDjAQ74VUrKBw16jGIofVskI88nvZsi4KHs s1kLR6mJpwTEnkKcRwe4wzH+Nm+OcFZw8Z6iaiaENnXTEVIPm0ByD2coQ07B4HQ1Ff1mONeUMME0 swYGEFoXFqUdH4FOFDMdElK4mn9LaUZMYpXD+ULYzBewSC5ogqxZioI7EuIHsWsIJ2iE7x6J1XOz TQxmMk3APANAIIKciSo6sdCFbfKAX5BhgUCKImjJONi4xQyG6TVgyeQ49wjVA2yCPbN8+IBiRvEX swkahZ0yaxqa9k3TIy2ADiC4ODlkprEBJLYdTQWIvQ8eqfeoxx/CUrYU5VFyt2UXnt+QiEOC2Cki eryEId/EUw4uyDfdxzSeYDVcXM2UpPil1mdojbY0jID1oYHjGuvmEZk4l3j+8Dwq7kbUKX6hPKNw whOLnpUb4MCjsnvRU58NldVborxN/Ovq9ditJzrxq768s+JfuNzovFghxUBGVN+2ZRJJYipkFdJc V29LX/xB7V8hvQxhmcI4yKIGivDa7xCwQJruYm9eKG+NgkSJ2w/fRA8Yaia8KIoOu5A7z2mz8h7I jaEc7xFDJjsD41TiJpnyScFDBZHTorjY8y6vA2RzIpAE7VCUcTw3yfIDluBAVOW3RHqs3fDNevs8 LhEzNBnk8YtzwySK4BN75acAFqCqBjIwpkoSjFn6VqftCYUSp6iEFx1Q6Ce6EnZTzu51Q+IT9BRJ SyNE5UAuJ5QuzzR/9If+QxegUCD3RuAnBpEXgJwSUUDxM9GRDCRxDb7ne3B3EiAoSKNCExMYBq3x JQhgGHegUxlxNOAyGcTxDMuERiEITeqCEgVzDScIBjmBLdqSa2OQfBihFmbTFiBzUcxUMKx2g7t3 FIVWCj2YJ6fUBXIWBhnAK6UmGdJxVG/0LzlYF+XShNZmaAz4EQnRN1PYBZZDJLhSL9wEAKeAWJHh gZ4RUgIlhukEMKH3LM6CXWsoCc+WBgNoGwNwDeTxWKsAhgBTZyKVh5CITvNjUBcQIAXUf9c0Jb3D gnlQiEBhDcpkT6+2fm9FMnNBLishTSQlF0fBT5EoFxtwDBrgAEcBQTD+VytgImV1QCESZVb+4RjS 8XeTdBAjA0gDgzKb8RnjZBX/EBrNWFLNSC4MQIta0RV9c4tUAAcGwIlqgIEZIUKMVVZmdQ2ZVApe aFEio0+umAp/9FSckVIqE1LTsVtvFFV2hRNTwgZbcFM5lTEXw1i0ATdElAuWwB1LExMcRYxuZRmn qAEopQql+FQa4DLVYBD+gY2A4IIa01d+JR4agwnJ4TQf8z1ulU9L1TzIuH4qCZGcFVr4FiGBGE9j ZgevAY4/81U505EZwTP5gQ6AxYMxtBFQs1alQAURmTK1MBSD1ULGEJMvMoit4FkEoBZpoTPVpZM7 aSbzgR4S0pNj5Tb+xtEficUxUAM1bpWUUpNYbOOUXWYzscGTtgGXHYkLMkQAlNUgf5dYlaWRSOMb pgATXWhPcKJYplEZGKlVE1c0u3FcWIkWjIEODrIRa0JYquUgoBiM3jFROHN5pbV+hKUL6oFQTimT 3HAgCFcMsBFWjIERvKZaDRKZeIkeD9SFMjQfxVANavlZi7EfbVOQqzciB5I3ZRVanjIA/DCWqMWV rmlaXSgciGUa1VAdS8NKi4EJ6jeaglhuJKJbpeAAvAkTH2dauIk2jYJY+gAu9yEc5REe7YCdPgiV LKIdpAAgpjGW5Rk0LERZxkAf5CGXGFEBmIh3jrMyXfFz0jU0D7J+D+ShmxkBiO75OPAZOHhDJ13z j353d0YWYSlzD26DoX8ToWHWSNoZolk0oiQqoCcKYyaaovi4oiwKFiD6onsQozKKWTWaXy56o5mY ozr6ZTPZo/73o0C6o0I6pD5qpCiKpCJapEpqBn02pE8KpIXgYFRapVZ6pViapdRjAEEAADs= ------=_NextPart_94915C5ABAF209EF376268C8 Content-Type: application/octet-stream; name="image002.gif" Content-Description: image002.gif Content-Id: <255661-2200241151539523805@sp.mailbr.com.br> R0lGODlheAAoAOUAABwHADgNADIrCFQVAG0cAVQuCWsuBlU/OG07IFdND3BPEXhqFHFGOJQoAakr AcYvAP49AI1YErJYDpltHKd1HZx2NLB8IcxaEP5DAcxoEPVtDpyIGqySHbWjH7iFJ76jINaMF/ON F/inHM6SJ+WeKtGyI/OuLNzCJPLRKf/qLbSQT7udZ8KZUtysV+yxSc+ra+S5b/7HTfjLcqSdkc+1 iNnDm+/Okv/hjtzNs+rXtPzktOjezPXoz////wAAAAAAACH/C05FVFNDQVBFMi4wAwEAAAAh+QQB DwA/ACwAAAAAeAAoAAUG/8CecEgsGo/IpHLJbDqf0Kh0Sq1ar9isdpvU8aQyl25Jq3HPWx0qdvs2 daa4LMmyWMzovFWG6rPdSXGCLjlFOR52dnhGOS8vepBHPH2UazaARDGCcSQjMEM0iBYepB42RI0j qh6FUzg4NDiRTmqVgjEygDaCfSadqjYvI6Ulw6Q2OTCqy6SPSzuvKysqE9UqsrNOfBoY3d0acbhw JN7dFMXLqiXr6x7p6SWlrUI8OTWOFtX6+h7Y2bQoynVz4GGTiXIQIAhAgU4duxInGo54GA8Rixd1 LFDYuHEfR1b/kvDgoaOkDhs2ZJATiCFBChSCIGCQCcHBBErrKlVaZ6JPRP92iexwHEpB38ZFZ3Io zYFShtMYLlwYnBqHG0sKKbKiCEGz24ASOsOKZcgO0UYOaNNy6FgNKRobUanKnRqQJQYHHbKmsDpT 4di/lH6W+KBWbceiE0p5UMHY0QsakMvUmLwUU5N6ceX20av3oN2WWmd2q3lTpwgRITRoCBEiLOsQ IGIbzkA7A+JqFC7o1k2UQtChL+ZNyRFGM2e+V/uE8JbwayURGhJKlx7iZZ++CSWoxV5T7YXpDiIM DWohuGUsOopTrfvZAQcU0UX7pRR9un0I1a9PL5A2g333adlngG8fmZdNDlIZtNxn3SRgQld3LdAL dzU9cN97AU2nAFoU3Af/IAffSTfARh4EF1J668En0AMCRcAVTRA4dxB4AnBQQn3SRaCfdBtyQCEE DVQjwXQNiEeBCjWcx0VJMmhGiUAxCuQAi/IJABGOEDywEAopPDidjhlKlwAH/t3XAFrYPZAAUXZ4 wEIZSkqhVJNURXTCnScs6A0BE9gF3gIlvCidAyCchpp9pXE35n0JNbBBmQnxl5Zv5IniZpJRJKMM CXJRtA5yGADAAaijNffBjYymCkFpWC6AXQMWNuqjdEEaRt6tiSxWRhKpvEPCr5zK9euKVoLAHXdW lqAqow2c0AeODwwpnQCvhliTAh6NhysFizmyqxIj1YPSuDBEZa6CAhEA/2gJCCU0kwOAimAfAQbU a++9BdAnXazN9ZlqAfvwo4K3NSgVZ1JSgSrqpzBON0AHgdqXL2cUB4YlrQvMeh+f+6gQEq+CrFgA O8Z2Ne06giaUAZcpcIWBaq3hdHGkaHqYccATXPMxETAEq+dMfKbTcELwsnOfavcNgGFEMxOQ1o+S 4lyNBSt8zIMLwN4oEADuLPPjw+z8eF8AHfjE8J9Pzyu11DpHYoOvIqvSQgsjzCzAO2InRMDIZq8D Kc1p0xoBKWvjbAENerz9zggUGIAAAwwcMAPPjN9rQASLU9CAAw00QMAAERSgADEPUVDA6adPwBEH oaNeAAWIDwFNZDRMo5yCHfqs4E8WdC/TAgzIHNxDDounE5wwy4BVUdeWAtVmb0VBXzUUO+ywxQvI COcE8ss4IhzxipUS+wuJcCuKMzzUQIMKHhg1lDU7bwG+90m8IIodsdOTUVBuyQ4Z+wGrgPXih4V6 XEYU+UPF/SzwhB3UrgITiMDuCJiNbyGBBnYYQRUmSMEOEsERHgyhCEdIwhKa8IQoTKEKV0jAIAAA IfkEAQ8APwAsBgADAGoAIQAFBv/An3BILBqPyCJP1ks6n9CodEoV9lw/l67K7Xq/RhkR1gSbh7vz eWu0qak7HG21sqzeYDZeGp//LBNDE3d7XzGFSDk/Lz8egUeDiJKTSRSMkj09PDxCW26UahRCl1SZ mTw6Ojk5NjIyMTFYoKCWXj2sslwms1wcRRQqLy80xDQ1x6ubPKamQ5k6NrlFu7zVTi85zUc9OjLS 1oWiUMLZmVPc3uDg2Npeeupm4kXY2fD2XR73+vv8/f7/AAMKHDhLBjWCUCqkmfQNIRQVOMq8yUHi SQmCFOQRsUBDIhgbFX9cdNhlhbkuPKSNRDIiYAUVKixYqOBkRcQqNloiMWBgiKJkQjrfmNoRR04x Ou2uBf0xYoQLGDZ+evkkZESJoCzy3TuZJEfTplBzcMLTgxSRGkYsfBipVR8PGmJ5eR3iYayQHG21 siCJqAeNHyw8Dnlhgchfvn0FE+Gh04NUxOq8toVs77C1IAAh+QQBDwA/ACwHAAMAZgAiAAUG/8Cf cEgsGo9IYq+XbDqf0Kh0mvvJmNOsdss18oauandcxNFoZC7PVQynpzjzSjWZqHBvbS8mNJmIMGJ5 RHgrPxYTRniDWTJGEiOMX2h0TouMmF0eP5tSgnlLXz86OmQmkZlHn1BLoaQ5NjIyMS58qbdEq06t Nmy4v05nNTU5OTw8rclHPTw5vkYmKCjA1Ec0yEtIPTq9R9PVYxQUTy85rVLbtODV5dldjuupBSrm WPH3+Pn6+/z9/v8AAwoEt2agFAs07L3R8cegk3FDVuBQ2AWGQyiJilSY2MUZkRMXs1jgmAUGqhIl fqQMSeTMijkqVFSYOWSkOyc5UDUx8EPFD19RuDi8SUaUohEaOomMGAHDhi5PRUZ0+vFiiIeV/Hi0 UMrUKSMbSaky4TH1KqdD98COaFHu6aAeVYXUUEjDghG764i5xdWDhlsWUVmCqzEVsGBwLzahOQwu 52LGBKkFAQAh+QQBDwA/ACwJAAQAYQAhAAUG/8CfcEgsGo9IYS/XSzqf0OguSnX2bEJZs8qt7nC0 lWo17ZqvRdjWzMb9wpWJcLJi27FInv3ppv0sckUTfntdPHiFUDk/Lz8eT4OJkkMmI5MWVR41kz89 PXqcoUMcHEWPXJ6ePDo6NjIyMaKTpU4TKlWeOTIusr1cKjU5Ozypa0eeOrtHJiYoKL7QRx4ewMXH h7xEJtGhFD+BSC9MxlY6sdzoQ+Ke6e1PFOBD7O709fb3+Pn6+/z9/v/QFgHsQkaSjIFUFIBbMc9M jmw/ShxZgHCOEYbkoMCwFLGiHYxRcnD0+ERFDjA0UqpUuaPhERhUCvywITBUiRP6RDoZAYPmnk9G Qk4JYZHkFCZ0L0YO4emTUw6gQzb9qHmEaq8cp0aMeCEOlK+njMhB/fHhlNdeLzxwzWEVHbGqQoOe FcWW5Ni4JANyxJsXWlq+fb8ShRYEACH5BAEPAD8ALAYABABlACIABQb/wJ9wSCwaj0hjL8lsOp84 HO1JpfJcMl11u91JV5XJD8ctG69DV87M3v1oYDGRzK5rj+t60g1XyY0rdHpmd4NONT8qFE6Bho6P dRQvbpBDPTyVmT+LQ5KGPaA6ojIyMZqnnYhmoD02pC6osUY0O6xsoDmwSSg/IrKZGSMqNKxLVaG6 Qya8v4Nif0MqOba3NkYmzYOcQxWg2d/g4eLj5OXm5+jp6uvs7XUy2O5PjZA6yfLzxoY2JPhcFTj0 mcnlj80Kb2VglBnBrpjDYlVsMBxSogSTF808kOOBsckIhXm4CPyhUUjHdRKRfIQRclCriT9eGKth YciHIlOa5WBRZMSISBc5WlbqkeMFpiI8hXiwKCSnLBo1PXh4QUOorJFDoCotqQoVDg8sgFpFx4Nh yR9dNXkpOOTF2bRsZeWYCDeuLLdj7aLKcfRUEAAh+QQBDwA/ACwGAAQAZQAiAAUG/8CfcEgsGo9I Yy7JbDqfvV3vSaXaTCZZdbuN4mgr1W/HLRuvQxLMzJ5+V5XilE23Idf0pBtsQdLmeWZ2gU9LLx5O f4SLjHQeNYCNPzw6kpY/E0QeS4GUOjYyMjFYl6VDKpxlOTIuWKSmsEUqNTk7PGw6rCZMKT+7sZIX FxQUHh6zWzw2rUUovcCBFD/SRRHEL6llOjFGKNCBHEUG2N/l5ufo6err7O3u7/Dx8vN0OS70VBMr kj1a+FX6GNn7x2VCQDo98BAsY3CfqjIt5OX4QqOiRYtkqCQkMmIEk0GxwqXrke2OnUhVSnoUwmJe jxcmc6Csd2jIIyE5EAnROaSkKUOSRV5g4zFT4AsLMIMOsfBhyC1gL38IzSHT3FMlOi30mRSrB0mZ RdtF3foj7CKvC3uuLJu2XFS2bcvlYGE27iW0poIAACH5BAEPAD8ALAkABABhACEABQb/wJ9wSCwa j0hhziRLOp/QaC9KdfJMQleuyuX2ejga7dst/3iuotbMnoZXKqEKxzbHktt60v1bWYwrdHpcOk2D UDw/ND9xToGHkD8oJZAeVS95hzw6OpGeRBQURS9dmzo5NjIyMS4xWJ+Hok4UpFQ2Lia5sLtcFGI1 NTk5PIlJPDm4r0UoKCkpvNBHoaEWFjTFRTq3ykIo0Z4cP7JIoZhUhWnf6kMUFubr8E4c40MqmfH4 +fr7/P3+/wADChy4CxvBKtYOdToYZcG4CY/M9IAxhNuQCQyF0BMyoYKgKsiEkMjIZoKFj04mkoxS QcwKOCpUVJg5oeZJJ/ecHPgxDNYIYUr9eixKAsNGjilsRgxROsQSkj8/nH7j0cJI0aOebDAdUsug EaTQetgQ0gLTUbC7XvyxUKPIUCEeoKKF1SPY2XzXjvRgUWTEXE9kSLYdwneluh613hr+lmOE4sXR enj9FAQAIfkEAQ8APwAsBwADAGYAIgAFBv/An3BILBqPSCJPlmw6n9CodOr6uXTTrHbLPTKFJlh3 bOztemSuzlQc2dJTM462WllWaHh2LUShiG56Rjs/ND92ZYJaMUYUJYpoOT8vHk55iphdLD+bUpd6 PDxYPzZvYyiPmUefUKI6OTk2MjIxMS4muKq6RaxOsbdsu8JOKi8vNMg0NcuwoTxHOja3SCgpKcPY RxYWFC+S0DLTRtfZYxwcT9vG31A64cHlwxTd7FtV8aoD9PX4/f7/AAMKHEiwoMGDCLH1+JIQyoQK hAT1uNewCbohE1TggMOvIhIKRiZYMMSlhykhqTxKmTBhhRYe90aM+EFCZREVKrZVYMkTY8txjU9M PmHwY8aPXphAptmxA8ecZHUsvTjiAoaNHEijnEQ5U0inHxYqDexRz2oOHlm5UCriocbRqUIsfAD7 Q2y8HjxonO2RlmPXus+G5LA75Gs2vn1V0bBgmAjcISRtYuPxt6PkXTlmJr4MicZmzpCwBQEAIfkE AQ8APwAsBgADAGoAIQAFBv/An3BILBqPyKLMpUs6n9CodEoV6kw/k6zK7Xq/RuzQlQObib3zOXaE qam9Hg5Ho6XfXxsevpv/VipDKjh7YGKFRjw/NT8vFkgehIiTlJBlkzw8Ok0/eluVahNCjFU5pjk2 NjKrMS4uJrCgshOkXDauh1UoslwURR7AHirDL8V1dTXJNaeKQjw5uEi7vNROFNcvl0Y5S7nVhRxR FuPZzVE63d/fFB7lYNrqZuFG7Nnx912++Pv8/f7/AAMKHEgQFLSCUiYEogQPYRSFkt70cONkREEO 84pMsLBCDQ8XQkg49ALRSw89QywmaSEwArAJMJ1spFHlZBIEDIbc2eNhT59jYzQAqRgHc8KKiE56 sBzSAoaNHDx2dnlBpASKEilV3tux40mcF09zxJF6JodWIjSLsOvZqN9YspNesP1hIa0zFkMeLRqJ iEfPukdyzP0Bl68aGrWM9Eg7orBhXj2KPebnmFIQACH5BAEPAD8ALAQAAgBvACEABQb/wJ9wSCwa j8ijTZdsOp/QqHSK5JlMMqp2y+0+Y0NXzksum5tMIgl2bru9JiGKOOK9ob0erncvZ39zQnFCL31C eT84NCsrFj8rfIaSbXw5NI+ORis7k4aDZC8eUIWTPDw6qEI2NjJ/nWYvY1w1NDQwMDIuui5XV69m okkqsl20IyMkXIG/XRMUzxQe0tIq1SwsL9m12zU1RDw2Lj+fRSgpKczpSc8qsVU5ueRDy+rMFhYv NMRROuHy9X0yUbDgIda+Ljb+ATzzLNvBhRCjUIhIsaLFixgzatzIsSNAOx6hTFjxi4e4kEg4FJmg AockkyiTbDgywcImNz1sDEkWc8oEfQ+XyuTc2YTNmWBeKEyIMmGkSy49jBKxwICBmQaSCu3Yse1H tQoVmjYd8jPolJy3bOTIkSfSmxFHLHz4MdfQDkW1VlRb8fRJW7dudA4pMSQYDaSZfjzsaYZFEaQ/ epAiAphxGxqZPICkPHmy5Tc5pG02ksPx4s9nODlhuzAIACH+c0NyZWF0ZWQgYnkgRWNsaXBzZSBE aWdpdGFsIEltYWdpbmcgqTE5OTggIEFsbCBSaWdodHMgUmVzZXJ2ZWQuIA0KQ29tbWVyY2lhbCBs aWNlbnNlIGF2YWlsYWJsZSBhdCB3d3cuZWNsaXBzZWQuY29tDQoAOw== ------=_NextPart_94915C5ABAF209EF376268C8 Content-Type: application/octet-stream; name="image003.gif" Content-Description: image003.gif Content-Id: <188792-2200241151539523806@sp.mailbr.com.br> R0lGODlhtQA+APMHABwXFVhVVISCe7W0tMa9vcbGvcfHx/39/cfHxwAAAAAAAAAAAAAAAAAAAAAA AAAAACH5BAEAAAgAIf4IR2lmIEx1YmUALAAAAAC1AD4AwxwXFVhVVISCe7W0tMa9vcbGvcfHx/39 /cfHxwAAAAAAAAAAAAAAAAAAAAAAAAAAAAT+EMlJq7046827/2AocoFhEMRpFOtZpCzxtjLs2jU9 x/ut470cD0j8GYdHH3KpbAqdwVmOEJiUVASAdsvtmqS0sCHQTbViYZwKrW6H0YbB9ooypdzstD7P x+79cGt+d1USdCddiQAlZHNfLnYmjVtmk3MDApiallcFiSVgKmNbApGWXGZdAytqbImrdWiWq3EB mgO3uJiJZr2FCHRaB8PExcOni8UGiQLKlMbLigHFcloGxsjE0VvXxMLG3MXIit3E4wDN5opc08VZ n3a/wQDG9QfnXOX37cPbA9Dr6BGrBkDfvWTDCCI8lqheOHUBH0L8JE7Rv4oR6ciz883exIj+7ABq ueixJEGDDCUeeCfNobWS9RqhhPmRAE17WTROAADnziBAgvrYQEOUlQlBrZAalaXqTa83fqIq7Rlp hqg1d3b4ZDH1atIUA+SBynqnhtkXZ1GgXau2bdq3bOG6jUt3rl25eOvmvau3L986ywhYicQyICyl Q0WdMpMHD9CtjxH/gUxZslDLkR1DvhwDwGAsIF8tVToKlQldqFOrxtVLtRrCkhzNWA0jNQyfuE+4 BnQHtU/dq4MPaOPZEOHQV+I0vVGpTGznKHIoTM6M1YvOWw4XVgSjaY3r3xE1BT9j1gmzZE93CdCL zUbxN/HZHEjKmZb52gJSU3lQC7/t7nD+4VJBGIFk0Dn89NfSfusE89k78YW2hUf+iKQIgwR6NIsx AnTooS0D6oNPPtg0WOB9JSLH0WdWseDiGUZVZkZmWP2U241R4bhdCU+Fh9ttsOmY43lx3UWakEgi MsEwOQWQyZO3QClllLhMaSWVV2apyZS5bFmll1qGCeaYV35J5pdiZhnAmt0suQx7R8Up55x01mnn nXjmqeeefPbp559htfMgZWsxN4QMhiZ6lqKFLupoo5Cqxaikj1IaKaKVYnrpoZxOiqkZYT0InHA9 TgYJAbZVReNlmmEmo6mtxgrrrI1BRcVgmEp4SFGvLfbIZkEFm9iwMGIaz5qHvUisssX+Nsvss1AV xautPW2Uq4T3HTXjts9RIgpVojwVbiTjHtltJ9yKqy6565bb7rvkknZqPLhyBNIpPGorhq+lYeuI CfBMy8IkPA7MXbcAHJZZdZqmIAAXh424TnsnWGtvRggDACQP/Wbbscd13MHOUfjOm0p2igXUHMqR YcdFKbkZ8DDLP4J3sbfVPthRSaeIqEg63qBo34X0vXSiRGWIE+LR6xy4DtAKRmThv7caBx9N56So BdQVDt1Q0Rl+JNAB2+y8zdJiN631ekyHhOF69FoNIdYZ2cd1dlN3gaHTCxWWoEJjBx121AHxTVHa IxkDONwV6xy4hhJCbSFJ/egH9oH+/Fgi+U0HqCSx0YgvRLjGazcYNzD2Rugvfvklbkx0sKewN+TP tH54MfzdeRBKda7NOtl5yqGT1X/6mZWcU9goBaJxfmfDUYzbGfuMg4BRFvLTZx/dVc37cGMoNGzE rLvjfmUrUnmYzLEbxoK63G+lNtYTG+uLoSmRQFavg8l20H9qcQjIwrbSVxlwtcooXTFf/eA3vf5h QWDoa5lkcDSu6+yvVhBMjHIOsCSykWFNIAyhCEdIwhKa8IQoTKEKV8jCFrrQhSSZwDJoUbwa2vCG OMyhDvu0Jg5KQGP6+o2+ukfEIRoxXkjknhKpt0TkObGISWSiFJ94xCZC0YpDdJj++H4lFSF60Stg TFcQxwibMnYxjGQ8oxjN+MU1qjGNbTxiFj7TgjxZz0ZSkdMdk8RHPPbxj34MJCAH+ZPGGYcr24EH F9nHFV/lhlW0WpUkXwXJSk7SVZlxDx3FAxLG6AhIjpygIEeZJD0S8pSkJCWQYlY11GkLW7Q4HqVs EEpZ2vJUt/wULgtJMG3lMga/PM8udUlMYA4TmMgkEiQM6cqb3esL10OLUXwlzWhaU5jLi9Eqb4MD iGkrCNvUpjizSc5rxsKc3GTFUxD5Hmc+E5fGNJhpbsi8OdmyJ/NkYhX3d7966pF/TvQnn+DXTk6u zg7pLI9zPoateYwsob2Yp0H+vVW2hEFTDxO1qFTM487QePE9LjMM4HpB0hbw63PSgF4XSjFM5VA0 ox4zj45COhIyzqymLvVXtpjXyuTAFG7bgRlCp7lQZNTJbyp92RhZoJDjTAxhtNijO2MpCJnqCXD5 kgIAfZpIRXbMp3I6KS/ohFSArbROvTzK4ubJ0TslkoZh9aaeTsFS5L2HCuypVTwW8cHm8TWEi2je mjT6yhEugivKWRMTPwhYGjJ2EbAbTuw+SD3GnpB6VPgrCCG7VxQSVhKNPd08VDeH0qFjaJQjm+US wh9kkIQT+0FbSgxk2q21LWlMG54r5waTcbAOq6jNm4AuV7rXvgwjsh0dOWr+e9rQtTalzBwt3RZU ufrYLrUVLS1xT4SfDU0kuSg1HNucO7jOmS66qZuu2iByN9fZbrysBd1su4uKE4FXQuJ122wTYRCM oXcFO6OdYfbT3oQJ1z+zw0g5vIA7vYFDvuFlboKy5jXo9jS9vUWOMijXtffq9wAnKdGCSaSNOd03 NPnV7n4nVGELgzTA9kAp6ezR4ereDsT8WYWNYcw5z+FXwkdLkGpV9F/e8sxfQrYxdleLY/l6TcgU bvDgInxbqME2QLoSLYaPrFOD1Jhz8S0vkz6sXNENgz8wicZMSDtjMI9Zt3QYzk0KMJw6o+rOdq6z jt2BizXTJA5yrgegUTDDwjzjOdB83jPnUDUcGL3o0WnQs59LEgdfiEp7sROkVNtnvU4/Un84cqQQ gcKG9JjaRWVB7J5u9D1TP9DScsPC9UI2GlbXyA0JPF9lSMOvVd4agRvL9Vbaxa4zPFKBXfmvMvtZ q+8JEzfJhPYg9bojB8ryJwvcNDxPVb/ypQFJ2o4uLMCgFT38aJzWLKmrAWk9oPJGnTHCdvJwVG6s +Prdw1aekMYwgCXJAa47DLjAB07wHZLhGiNIuMIXzvCGO/zhEogAADs= ------=_NextPart_94915C5ABAF209EF376268C8-- From neal@metaslash.com Mon Apr 15 16:58:37 2002 From: neal@metaslash.com (Neal Norwitz) Date: Mon, 15 Apr 2002 11:58:37 -0400 Subject: [Python-Dev] Re: Deprecating divmod() // % for complex References: <200204151244.g3FCiW116681@pcp742651pcs.reston01.va.comcast.net> <3CBACC90.B7FACD76@metaslash.com> <200204151310.g3FDAsF17235@pcp742651pcs.reston01.va.comcast.net> <3CBAD2BA.FA4B695C@metaslash.com> <15546.61154.973803.448931@grendel.zope.com> Message-ID: <3CBAF8AD.7825977E@metaslash.com> "Fred L. Drake, Jr." wrote: > > Neal Norwitz writes: > > So the APIs and doc can be removed. Here's the current list, > > found by a lot of grepping in the Python modules: > > The docs cannot be removed when something becomes deprecated, only > when it is removed. Deprecated features still need to be documented > since someone will need to modify code to not use the deprecated > feature. This requires understanding the deprecated material as well > as whatever is used to supplant it. I agree. My point (which wasn't clear) was how does one know when to remove a deprecated feature from X years ago. Right now, it takes a lot of work to grep through the code & docs. Neal From bernie@3captus.com Mon Apr 15 17:43:57 2002 From: bernie@3captus.com (Bernard Yue) Date: Mon, 15 Apr 2002 10:43:57 -0600 Subject: [Python-Dev] timeoutsocket.py Message-ID: <3CBB034D.BE3B79C2@3captus.com> Hi Guys, What is the status of integrating timeoutsocket.py into standard library? Are there any body responsible for it currently? If not I would like to give it a shot. Bernie -- There are three schools of magic. One: State a tautology, then ring the changes on its corollaries; that's philosophy. Two: Record many facts. Try to find a pattern. Then make a wrong guess at the next fact; that's science. Three: Be aware that you live in a malevolent Universe controlled by Murphy's Law, sometimes offset by Brewster's Factor; that's engineering. -- Robert A. Heinlein From pedroni@inf.ethz.ch Mon Apr 15 17:41:12 2002 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Mon, 15 Apr 2002 18:41:12 +0200 Subject: [Python-Dev] rexec question References: <15544.46543.51666.462318@jade.cabochon.com> <200204151424.g3FEOZY12635@localhost.localdomain> <006101c1e491$851dd440$6d94fea9@newmexico> <200204151557.g3FFuxY12921@localhost.localdomain> Message-ID: <00ac01c1e49c$5a979840$6d94fea9@newmexico> what's the general python-dev opinion about rexec security enforcement? can it be trusted? it is up e.g. to java security? Thanks. From jeremy@zope.com Mon Apr 15 17:58:52 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 15 Apr 2002 12:58:52 -0400 Subject: [Python-Dev] rexec question In-Reply-To: <00ac01c1e49c$5a979840$6d94fea9@newmexico> References: <15544.46543.51666.462318@jade.cabochon.com> <200204151424.g3FEOZY12635@localhost.localdomain> <006101c1e491$851dd440$6d94fea9@newmexico> <200204151557.g3FFuxY12921@localhost.localdomain> <00ac01c1e49c$5a979840$6d94fea9@newmexico> Message-ID: <15547.1740.887399.669503@slothrop.zope.com> >>>>> "SP" == Samuele Pedroni writes: SP> what's the general python-dev opinion about rexec security SP> enforcement? SP> can it be trusted? it is up e.g. to java security? No. There's been no serious maintenance of rexec since work on the Knowbot system stopped, AFAIK. New-style types, e.g., don't have any of the same checks that classic classes have. There are really two parts to rexec: The basic checks in the interpreter to limit access to certain objects, and the rexec import hooks and related mechanism. The Bastion object plus basic checks in the interpreter limit access to an object (can't access __dict__, classes are immutable, etc.) in order to achieve some kind of enforced encapsulation; Java's type system provides this automatically. Jim Fulton has suggested use proxies to implement security (mislabelled, I think, capabilities): http://www.zope.org//Wikis/DevSite/Projects/ComponentArchitecture/CapabilityBasedSecurity I really like this proposal because it takes all the various rexec checks in the interpreter and pushes them out to a proxy object that can implement exactly the checks needed. I'd have more trust in this mechanism, because there is less code to inspect to review the encapsulation mechanism. Jeremy From thomas.heller@ion-tof.com Mon Apr 15 18:51:52 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Mon, 15 Apr 2002 19:51:52 +0200 Subject: [Python-Dev] Stale (?) CVS lock Message-ID: <08b901c1e4a6$3987a6e0$475afea9@thomasnotebook> There seems to be a stale lock in the CVS rep on SF: cvs server: [10:42:33] waiting for anoncvs_python's lock in /cvsroot/python/python/dist/src/Doc/tools Thomas From David Abrahams" Message-ID: <113101c1e4a8$9a73bf00$6801a8c0@boostconsulting.com> FWIW, many boosters (but not me) were having problems with a stale lock on our SF CVS this morning as well. Might be system-wide. -Dave ----- Original Message ----- From: "Thomas Heller" To: Sent: Monday, April 15, 2002 12:51 PM Subject: [Python-Dev] Stale (?) CVS lock > There seems to be a stale lock in the CVS rep on SF: > > cvs server: [10:42:33] waiting for anoncvs_python's lock in /cvsroot/python/python/dist/src/Doc/tools > > Thomas > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev From guido@python.org Mon Apr 15 19:38:10 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 15 Apr 2002 14:38:10 -0400 Subject: [Python-Dev] timeoutsocket.py In-Reply-To: Your message of "Mon, 15 Apr 2002 10:43:57 MDT." <3CBB034D.BE3B79C2@3captus.com> References: <3CBB034D.BE3B79C2@3captus.com> Message-ID: <200204151838.g3FIcAf27177@pcp742651pcs.reston01.va.comcast.net> > What is the status of integrating timeoutsocket.py into standard > library? Are there any body responsible for it currently? If not I > would like to give it a shot. Isn't one of the problems that the timeout is global? Also, I'd expect that if you did this as a standard feature, the implementation would be completely different (e.g. integrate it in the C socket code). I think it would be great if timeouts were added to sockets (and *maybe* the global timeout is even a good thing). But I'd like to see a design first rather than code. --Guido van Rossum (home page: http://www.python.org/~guido/) From bernie@3captus.com Mon Apr 15 20:09:50 2002 From: bernie@3captus.com (Bernard Yue) Date: Mon, 15 Apr 2002 13:09:50 -0600 Subject: [Python-Dev] timeoutsocket.py References: <3CBB034D.BE3B79C2@3captus.com> <200204151838.g3FIcAf27177@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CBB257E.F072C4C4@3captus.com> Guido van Rossum wrote: > > What is the status of integrating timeoutsocket.py into standard > > library? Are there any body responsible for it currently? If not I > > would like to give it a shot. > > Isn't one of the problems that the timeout is global? > It is indeed. IMO, Timeout should be per-socket based. > > Also, I'd expect that if you did this as a standard feature, the > implementation would be completely different (e.g. integrate it in the > C socket code). > Yes, because I find that there is a need for that (I needed it quite frequently). > > I think it would be great if timeouts were added to sockets (and > *maybe* the global timeout is even a good thing). But I'd like to see > a design first rather than code. > So it looks like nobody is working on it, I will try to create a design and summited it for comment. > > --Guido van Rossum (home page: http://www.python.org/~guido/) Bernie -- There are three schools of magic. One: State a tautology, then ring the changes on its corollaries; that's philosophy. Two: Record many facts. Try to find a pattern. Then make a wrong guess at the next fact; that's science. Three: Be aware that you live in a malevolent Universe controlled by Murphy's Law, sometimes offset by Brewster's Factor; that's engineering. -- Robert A. Heinlein From thomas.heller@ion-tof.com Mon Apr 15 20:22:22 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Mon, 15 Apr 2002 21:22:22 +0200 Subject: [Python-Dev] Re: [Doc-SIG] [development doc updates] References: <20020415185027.336EC18EAD1@grendel.zope.com> Message-ID: <0a3a01c1e4b2$de666a50$475afea9@thomasnotebook> From: "Fred L. Drake" > The development version of the documentation has been updated: > > http://www.python.org/dev/doc/devel/ > > Add a lot of new information about the PyTypeObject structure to the > Python/C API Reference Manual. > There's something wrong with the index. Looking at http://www.python.org/dev/doc/devel/api/genindex.html#letter-o ob_size and ob_refcnt are listed as 'int member' (for example). OTOH, it's great to have this stuff in the manuals finally. Thomas From guido@python.org Mon Apr 15 20:31:01 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 15 Apr 2002 15:31:01 -0400 Subject: [Python-Dev] timeoutsocket.py In-Reply-To: Your message of "Mon, 15 Apr 2002 13:09:50 MDT." <3CBB257E.F072C4C4@3captus.com> References: <3CBB034D.BE3B79C2@3captus.com> <200204151838.g3FIcAf27177@pcp742651pcs.reston01.va.comcast.net> <3CBB257E.F072C4C4@3captus.com> Message-ID: <200204151931.g3FJV1327452@pcp742651pcs.reston01.va.comcast.net> > So it looks like nobody is working on it, I will try to create a design > and summited it for comment. Excellent! --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Mon Apr 15 20:29:35 2002 From: skip@pobox.com (Skip Montanaro) Date: Mon, 15 Apr 2002 14:29:35 -0500 Subject: [Python-Dev] timeoutsocket.py In-Reply-To: <200204151838.g3FIcAf27177@pcp742651pcs.reston01.va.comcast.net> References: <3CBB034D.BE3B79C2@3captus.com> <200204151838.g3FIcAf27177@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15547.10783.567905.224673@12-248-41-177.client.attbi.com> >> What is the status of integrating timeoutsocket.py into standard >> library? Are there any body responsible for it currently? If not I >> would like to give it a shot. Guido> Isn't one of the problems that the timeout is global? That's the easiest way to use it, but you can specify timeouts on a per-socket basis. From the module docstring: import timeoutsocket import httplib H = httplib.HTTP("www.python.org") H.sock.set_timeout(30) The author, Tim O'Malley, is probably the best person to address issues with this package, so I've added him to the cc list in case he's not a python-dev subscriber. Skip From fdrake@acm.org Mon Apr 15 20:37:00 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 15 Apr 2002 15:37:00 -0400 Subject: [Python-Dev] Re: [Doc-SIG] [development doc updates] In-Reply-To: <0a3a01c1e4b2$de666a50$475afea9@thomasnotebook> References: <20020415185027.336EC18EAD1@grendel.zope.com> <0a3a01c1e4b2$de666a50$475afea9@thomasnotebook> Message-ID: <15547.11228.732372.267150@grendel.zope.com> Thomas Heller writes: > There's something wrong with the index. Looking at > http://www.python.org/dev/doc/devel/api/genindex.html#letter-o > ob_size and ob_refcnt are listed as 'int member' (for example). Hey, Python ints are real object & can have attributes! ;-) Seriously, fixed now. Thanks for the sharp eye! > OTOH, it's great to have this stuff in the manuals finally. Yeah; I'd like to generally improve the C API coverage substantially, and add some more tutorial material to the Extending & Embedding manual as well. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From aahz@pythoncraft.com Mon Apr 15 21:00:30 2002 From: aahz@pythoncraft.com (Aahz) Date: Mon, 15 Apr 2002 16:00:30 -0400 Subject: [Python-Dev] Building on Windows (was Re: A "new" kind of leak) In-Reply-To: <200204151257.g3FCvN217107@pcp742651pcs.reston01.va.comcast.net> References: <20020414114837.A21131@glacier.arctrix.com> <2mr8lhmejv.fsf@starship.python.net> <200204151257.g3FCvN217107@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020415200030.GA19740@panix.com> On Mon, Apr 15, 2002, Guido van Rossum wrote: > > Don't people use TAGS or tags any more? Works for me (in Emacs). > Just say "make TAGS" every once in a while. What's Emacs? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ What if there were no rhetorical questions? From sholden@holdenweb.com Mon Apr 15 21:10:41 2002 From: sholden@holdenweb.com (Steve Holden) Date: Mon, 15 Apr 2002 16:10:41 -0400 Subject: [Python-Dev] timeoutsocket.py References: <3CBB034D.BE3B79C2@3captus.com> <200204151838.g3FIcAf27177@pcp742651pcs.reston01.va.comcast.net> <048901c1e4b2$8a104c00$0400000a@COMPUTER> <200204151935.g3FJZgJ27526@pcp742651pcs.reston01.va.comcast.net> Message-ID: <04cb01c1e4b9$9ee190b0$0400000a@COMPUTER> Guido wrote: > > [Bernard Yue] > > > > What is the status of integrating timeoutsocket.py into standard > > > > library? Are there any body responsible for it currently? If not I > > > > would like to give it a shot. > > > > [Guido] > > > Isn't one of the problems that the timeout is global? > > > > [Me] > > While there *is* a global timeout, it can be changed locally before > > calling the library modsule that creates a given socket. The normal > > mode of usage, of course, is to set a timeout after explicitly > > creating each timeoutsocket. Clearly this is no use with opaque > > code that creates sockets. > > > > > Also, I'd expect that if you did this as a standard feature, the > > > implementation would be completely different (e.g. integrate it in the > > > C socket code). > > > > > Hmm. That would increase the complexity of the code, given that > > socket.c has nested "#ifdef"s and many platform dependencies that > > timeoutsocket.py does without. I wouldn't presume to comment on the > > difficulty level - it would certainly scare me away, but then I've > > avoided C for the past few years as a matter of choice. > > Yeah, but the shim layer added by timeoutsocket.py is *also* something > I could do without (especially since there's already a shim layer > added by socket.py). That's something I hadn't thought of. Perhaps Bernard could integrate the timeoutsocket.py code into socket.py? This would need less work than a C rewrite, even with a redesign. > > > I think it would be great if timeouts were added to sockets (and > > > *maybe* the global timeout is even a good thing). But I'd like to see > > > a design first rather than code. > > > > > It could certainly be helpful to network newbies if it were more > > difficult to write a program that hangs indefinitely. > > But do network newbies really care? They can just ^C their program. > I'd have thought that this is mostly useful for the gurus who want to > write long-running programs that never get stuck. > I guess the newbies only care when they don't understand *why* they have to ^C. A timeoutsocket error message might provide a helpful explanation. Certainly the main value is for production code. > (Anyway, I'd rather see this discussion on python-dev, where Bernard > Yue can see it and others can chime in.) > Sorry, my bad for not replying to the list. regards Steve From bernie@3captus.com Mon Apr 15 21:37:53 2002 From: bernie@3captus.com (Bernard Yue) Date: Mon, 15 Apr 2002 14:37:53 -0600 Subject: [Python-Dev] timeoutsocket.py References: <3CBB034D.BE3B79C2@3captus.com> <200204151838.g3FIcAf27177@pcp742651pcs.reston01.va.comcast.net> <15547.10783.567905.224673@12-248-41-177.client.attbi.com> Message-ID: <3CBB3A20.35C8132F@3captus.com> Skip Montanaro wrote: > >> What is the status of integrating timeoutsocket.py into standard > >> library? Are there any body responsible for it currently? If not I > >> would like to give it a shot. > > Guido> Isn't one of the problems that the timeout is global? > > That's the easiest way to use it, but you can specify timeouts on a > per-socket basis. From the module docstring: > > import timeoutsocket > import httplib > H = httplib.HTTP("www.python.org") > H.sock.set_timeout(30) > > The author, Tim O'Malley, is probably the best person to address issues with > this package, so I've added him to the cc list in case he's not a python-dev > subscriber. > > Skip Just have a look at socket.py and timeoutsocket.py (version 1.21, assuming the lastest). Looks like that class _socketobject and class Timeoutsocket can be merged with no conflict. Now it comes the question: 1. Shall we merge socket.py and timeoutsocket.py? 2. If yes to Q1, how are we going to due with the copyright issue? 3. If no to Q1, are we going to just add timeoutsocket.py to standard library (Though there the copyright issue stays)? Bernie -- In Scotland, a new game was invented. It was entitled Gentlemen Only Ladies Forbidden.... and thus the word GOLF entered into the English language. From martin@v.loewis.de Mon Apr 15 21:36:02 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 15 Apr 2002 22:36:02 +0200 Subject: [Python-Dev] Re: Deprecating divmod() // % for complex In-Reply-To: <200204151502.g3FF2AV25900@pcp742651pcs.reston01.va.comcast.net> References: <200204151244.g3FCiW116681@pcp742651pcs.reston01.va.comcast.net> <3CBACC90.B7FACD76@metaslash.com> <200204151310.g3FDAsF17235@pcp742651pcs.reston01.va.comcast.net> <15546.59644.504199.808337@grendel.zope.com> <200204151502.g3FF2AV25900@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > > If that were the only issue, what's the point of maintaining a list in > > the PEP at all? > > Beats me. Maybe because the PEP author thought deprecating whole > modules was a heavier kind of operation? Maybe because he didn't > realize other things could be deprecated as well? Maybe because he > planned a separate PEP for those? He thought that it was indeed desirable to have a consolidated list of all deprecated stuff, together with a rationale, so that people could easily oppose the rationale. That idea never caught on. It's not really necessary to maintain that consolidated list *in* the PEP; any other place would be fine as well - but it needs to be a single list, and it needs to be maintained. Regards, Martin From barry@zope.com Mon Apr 15 21:39:39 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 15 Apr 2002 16:39:39 -0400 Subject: [Python-Dev] DeprecationWarnings in the regression tests Message-ID: <15547.14987.264776.590478@anthem.wooz.org> I wonder if we shouldn't just add warnings.filterwarnings('ignore', '', DeprecationWarning) at the top of regrtest.py? Seems like with the proliferation of deprecations, I'm finding I want to add something like this (with the module parameter added to each file that tests a newly deprecated feature. If this is going to be SOP for the test suite, we might as well kill them all with one fell swoop. -Barry From guido@python.org Mon Apr 15 21:46:55 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 15 Apr 2002 16:46:55 -0400 Subject: [Python-Dev] Re: Deprecating divmod() // % for complex In-Reply-To: Your message of "15 Apr 2002 22:36:02 +0200." References: <200204151244.g3FCiW116681@pcp742651pcs.reston01.va.comcast.net> <3CBACC90.B7FACD76@metaslash.com> <200204151310.g3FDAsF17235@pcp742651pcs.reston01.va.comcast.net> <15546.59644.504199.808337@grendel.zope.com> <200204151502.g3FF2AV25900@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204152046.g3FKktt27767@pcp742651pcs.reston01.va.comcast.net> [Somebody] > > > If that were the only issue, what's the point of maintaining a > > > list in the PEP at all? [Guido] > > Beats me. Maybe because the PEP author thought deprecating whole > > modules was a heavier kind of operation? Maybe because he didn't > > realize other things could be deprecated as well? Maybe because he > > planned a separate PEP for those? [The PEP author] > He thought that it was indeed desirable to have a consolidated list of > all deprecated stuff, together with a rationale, so that people could > easily oppose the rationale. That idea never caught on. > > It's not really necessary to maintain that consolidated list *in* the > PEP; any other place would be fine as well - but it needs to be a > single list, and it needs to be maintained. Well, it seems we finally all agree on what needs to be done. I'm guessing it's up to Fred to implement it. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 15 21:48:30 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 15 Apr 2002 16:48:30 -0400 Subject: [Python-Dev] timeoutsocket.py In-Reply-To: Your message of "Mon, 15 Apr 2002 14:37:53 MDT." <3CBB3A20.35C8132F@3captus.com> References: <3CBB034D.BE3B79C2@3captus.com> <200204151838.g3FIcAf27177@pcp742651pcs.reston01.va.comcast.net> <15547.10783.567905.224673@12-248-41-177.client.attbi.com> <3CBB3A20.35C8132F@3captus.com> Message-ID: <200204152048.g3FKmUP27779@pcp742651pcs.reston01.va.comcast.net> > 2. If yes to Q1, how are we going to due with the copyright issue? > 3. If no to Q1, are we going to just add timeoutsocket.py to > standard library (Though there the copyright issue stays)? Let's ask Tim O'Malley if he's willing to assign the code the the PSF using something like this form: http://www.python.org/psf/psf-contributor-agreement.html --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 15 22:21:27 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 15 Apr 2002 17:21:27 -0400 Subject: [Python-Dev] DeprecationWarnings in the regression tests In-Reply-To: Your message of "Mon, 15 Apr 2002 16:39:39 EDT." <15547.14987.264776.590478@anthem.wooz.org> References: <15547.14987.264776.590478@anthem.wooz.org> Message-ID: <200204152121.g3FLLSt02566@pcp742651pcs.reston01.va.comcast.net> > I wonder if we shouldn't just add > > warnings.filterwarnings('ignore', '', DeprecationWarning) > > at the top of regrtest.py? Seems like with the proliferation of > deprecations, I'm finding I want to add something like this (with the > module parameter added to each file that tests a newly deprecated > feature. If this is going to be SOP for the test suite, we might as > well kill them all with one fell swoop. No, because a test might use some feature that is deprecated, not as the feature to be tested, but as a tool for the testing. (E.g. a test function might use "from M import *".) With your approach we would never hear about this deprecation until the feature is finally removed. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Mon Apr 15 22:25:38 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 15 Apr 2002 17:25:38 -0400 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Objects complexobject.c,2.57,2.58 In-Reply-To: <200204151244.g3FCiW116681@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido, on deprecating % for complex] > ... > However, I'm wondering what to do after we unify all numeric types > (*if* we ever decide to do that, which isn't clear). At that point, > should these operators be allowed as long as the imaginary part is > zero? Not if you ask me. Because two mathematically equivalent expressions can yield different results when evaluated via computer fp arithmetic, it's impossible to guess whether a computed complex result "really" has a non-zero imaginary part (indeed, under the constructive reals, answering that question is equivalent to solving the halting problem -- it's not just the finitude of fp arithmetic that's to blame for the uncertainty, and the uncertainty runs deep). In practical terms, this means the imaginary part of a given complex expression can "round to 0", or not, depending on accidents of implementation, evaluation order, platform library implementation, and fp hardware. If the language can't define when a complex expression is safe to use in a non-complex context, or even guarantee that a given fixed complex expression will yield the same "is or isn't?" answer across platforms or releases, the best you can say about it is "implementation defined -- use at your own risk". I'd rather force the issue in my code by explicitly tossing the imaginary component. Note that this implies all numeric types should grow .real and .imag attributes (else code can't be generic, or has to resort to type checks, or maybe just try: x = x.real except AttributeError: pass is good enough). Another possibility for x-platform x-release consistency is to keep and maintain an "exact?" bit with numbers, and barf on a complex value with a non-exact 0 imaginary part. This isn't really useful, though, unless e.g. the exact bit remains set for 24./4. but not 24./5. (assuming the inputs are exact), and then carefully defining when exactness is and isn't preserved is a major undertaking. BTW, I'm more a fan of Common Lisp's concessions to numeric reality than to Scheme's attempt to wish numeric unpleasantness away. CL generally allows for "upward" conversion by magic, but generally never "downward". So, e.g., CL's divmod equivalent allows int, ratio and float arguments, but never complex. That's a recipe for x-platform sanity . See http://www.math.uio.no/cltl/clm/node121.html#SECTION001600000000000000000 and especially http://www.math.uio.no/cltl/clm/node130.html#SECTION001660000000000000000 From nas@python.ca Mon Apr 15 22:56:48 2002 From: nas@python.ca (Neil Schemenauer) Date: Mon, 15 Apr 2002 14:56:48 -0700 Subject: [Python-Dev] float.decode()? Message-ID: <20020415145648.A25455@glacier.arctrix.com> Common Lisp has a function called decode-float: (decode-float float) => significand, exponent, sign computes three values that characterize float. The first value is of the same type as float and represents the significand. The second value represents the exponent to which the radix (notated in this description by b) must be raised to obtain the value that, when multiplied with the first result, produces the absolute value of float. If float is zero, any integer value may be returned, provided that the identity shown for scale-float holds. The third value is of the same type as float and is 1.0 if float is greater than or equal to zero or -1.0 otherwise. How hard would it be to write a method like this for Python floats? I think it could go a long way in helping people understand how floating point works. Neil From jmiller@stsci.edu Mon Apr 15 22:54:07 2002 From: jmiller@stsci.edu (Todd Miller) Date: Mon, 15 Apr 2002 17:54:07 -0400 Subject: [Python-Dev] Inplace multiply by float Message-ID: <3CBB4BFF.4050707@stsci.edu> This is a multi-part message in MIME format. --------------020100030200060800050708 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Hi, Forgive me for asking this question again, but when I import the following from "example.py": class test1(object): def __init__(self): object.__init__(self) def __imul__(self, other): print "no luck here!" class test2: def __imul__(self, other): print "cool!" t = test1() u = test2() t *= 1.0 u *= 1.0 The following happens: Python 2.3a0 (#6, Apr 15 2002, 17:34:04) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-98)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import example Traceback (most recent call last): File "", line 1, in ? File "example.py", line 18, in ? t *= 1.0 TypeError: can't multiply sequence to non-int >>> I take this to mean that python does not (currently) support inplace multiply by floats for new style classes. Is this a bug? A permanent limitation? Regards, Todd -- Todd Miller jmiller@stsci.edu STSCI / SSG (410) 338 4576 --------------020100030200060800050708 Content-Type: text/plain; name="example.py" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="example.py" class test1(object): def __init__(self): object.__init__(self) def __imul__(self, other): print "no luck here!" class test2: def __imul__(self, other): print "cool!" t = test1() u = test2() t *= 1.0 u *= 1.0 --------------020100030200060800050708-- From barry@zope.com Mon Apr 15 22:55:38 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 15 Apr 2002 17:55:38 -0400 Subject: [Python-Dev] DeprecationWarnings in the regression tests References: <15547.14987.264776.590478@anthem.wooz.org> <200204152121.g3FLLSt02566@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15547.19546.808399.795215@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> No, because a test might use some feature that is deprecated, GvR> not as the feature to be tested, but as a tool for the GvR> testing. (E.g. a test function might use "from M import *".) GvR> With your approach we would never hear about this deprecation GvR> until the feature is finally removed. By the same token then, we probably shouldn't be suppressing the warnings at the top of certain files; the effect would be the same (but limited to only those files that /do/ test deprecated features). Hmm, what about making this controllable via a command line option to regrtest? Then what should the default be? Probably, "don't suppress" and then most people will just have to ignore the warnings. In that case, I'd argue that the switch should be "suppress" by default in the final releases, so as not to confuse end users. -Barry From aleax@aleax.it Mon Apr 15 23:05:39 2002 From: aleax@aleax.it (Alex Martelli) Date: Tue, 16 Apr 2002 00:05:39 +0200 Subject: [Python-Dev] float.decode()? In-Reply-To: <20020415145648.A25455@glacier.arctrix.com> References: <20020415145648.A25455@glacier.arctrix.com> Message-ID: <02041600053907.29389@arthur> On Monday 15 April 2002 23:56, Neil Schemenauer wrote: > Common Lisp has a function called decode-float: > > (decode-float float) => significand, exponent, sign ... > How hard would it be to write a method like this for Python floats? I You mean something like: import math def decode_float(f): m, e = math.frexp(f) mabs = math.fabs(m) return mabs, e, mabs and (m/mabs) or am I missing something? Alex From guido@python.org Mon Apr 15 23:10:49 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 15 Apr 2002 18:10:49 -0400 Subject: [Python-Dev] DeprecationWarnings in the regression tests In-Reply-To: Your message of "Mon, 15 Apr 2002 17:55:38 EDT." <15547.19546.808399.795215@anthem.wooz.org> References: <15547.14987.264776.590478@anthem.wooz.org> <200204152121.g3FLLSt02566@pcp742651pcs.reston01.va.comcast.net> <15547.19546.808399.795215@anthem.wooz.org> Message-ID: <200204152210.g3FMAoq02716@pcp742651pcs.reston01.va.comcast.net> > GvR> No, because a test might use some feature that is deprecated, > GvR> not as the feature to be tested, but as a tool for the > GvR> testing. (E.g. a test function might use "from M import *".) > > GvR> With your approach we would never hear about this deprecation > GvR> until the feature is finally removed. > > By the same token then, we probably shouldn't be suppressing the > warnings at the top of certain files; the effect would be the same > (but limited to only those files that /do/ test deprecated features). Actually, the filterwarnings() calls are usually pretty selective, by specifying a substring that must occur in the suppressed warning. (A few don't do this, and ought to be fixed.) > Hmm, what about making this controllable via a command line option to > regrtest? Then what should the default be? Probably, "don't > suppress" and then most people will just have to ignore the warnings. > In that case, I'd argue that the switch should be "suppress" by > default in the final releases, so as not to confuse end users. Why do you want this? Did you get a particular warning that bugs you? If not, let's please call a YAGNI on this. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Mon Apr 15 23:14:28 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 15 Apr 2002 18:14:28 -0400 Subject: [Python-Dev] float.decode()? In-Reply-To: <20020415145648.A25455@glacier.arctrix.com> Message-ID: [Neil Schemenauer] > Common Lisp has a function called decode-float: > > (decode-float float) => significand, exponent, sign > > computes three values that characterize float. The first value is of > the same type as float and represents the significand. The second > value represents the exponent to which the radix (notated in this > description by b) must be raised to obtain the value that, when > multiplied with the first result, produces the absolute value of > float. If float is zero, any integer value may be returned, provided > that the identity shown for scale-float holds. The third value is of > the same type as float and is 1.0 if float is greater than or equal > to zero or -1.0 otherwise. > > How hard would it be to write a method like this for Python floats? I > think it could go a long way in helping people understand how floating > point works. Python's math.frexp() already exposes a very similar standard C function. If you take the terms as defined above, it's math.frexp(float) => significand * sign, exponent >>> math.frexp(18) (0.5625, 5) >>> math.frexp(-18.5) (-0.578125, 5) >>> Separating out the sign is a nice little convenience, reflecting common use (most uses of frexp() immediately test-and-branch on the significand's sign bit right after!). "decode-float" sounds like some Unicode function, while "frexp" is wonderfully mnemonic . From guido@python.org Mon Apr 15 23:19:23 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 15 Apr 2002 18:19:23 -0400 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Objects complexobject.c,2.57,2.58 In-Reply-To: Your message of "Mon, 15 Apr 2002 17:25:38 EDT." References: Message-ID: <200204152219.g3FMJN302807@pcp742651pcs.reston01.va.comcast.net> [Tim] > Not if you ask me. Because two mathematically equivalent > expressions can yield different results when evaluated via computer > fp arithmetic, it's impossible to guess whether a computed complex > result "really" has a non-zero imaginary part (indeed, under the > constructive reals, answering that question is equivalent to solving > the halting problem -- it's not just the finitude of fp arithmetic > that's to blame for the uncertainty, and the uncertainty runs deep). I presume that you also object against allowing certain "int-requiring" operations (like use as sequence index) on floats when the fraction is zero. Unified numbers are rapidly becoming less attractive this way... > In practical terms, this means the imaginary part of a given complex > expression can "round to 0", or not, depending on accidents of > implementation, evaluation order, platform library implementation, > and fp hardware. If the language can't define when a complex > expression is safe to use in a non-complex context, or even > guarantee that a given fixed complex expression will yield the same > "is or isn't?" answer across platforms or releases, the best you can > say about it is "implementation defined -- use at your own risk". > I'd rather force the issue in my code by explicitly tossing the > imaginary component. Note that this implies all numeric types > should grow .real and .imag attributes (else code can't be generic, > or has to resort to type checks, or maybe just > > try: > x = x.real > except AttributeError: > pass > > is good enough). To me, unified numbers means that there's only one numeric type, and it always has the same properties. That means that all numbers must have an imag field, which is zero when it's not a complex number. But it seems you want some other indicator that says it's never been part of a calculation involving complex numbers... > Another possibility for x-platform x-release consistency is to keep > and maintain an "exact?" bit with numbers, and barf on a complex > value with a non-exact 0 imaginary part. This isn't really useful, > though, unless e.g. the exact bit remains set for 24./4. but not > 24./5. (assuming the inputs are exact), and then carefully defining > when exactness is and isn't preserved is a major undertaking. Unified numbers are becoming even less attractive... > BTW, I'm more a fan of Common Lisp's concessions to numeric reality > than to Scheme's attempt to wish numeric unpleasantness away. CL > generally allows for "upward" conversion by magic, but generally > never "downward". So, e.g., CL's divmod equivalent allows int, > ratio and float arguments, but never complex. That's a recipe for > x-platform sanity . See > > http://www.math.uio.no/cltl/clm/node121.html#SECTION001600000000000000000 > > and especially > > http://www.math.uio.no/cltl/clm/node130.html#SECTION001660000000000000000 Too much to read. :-( But if the end result is that users will write trunc() or round() calls whenever they have a float value that they believe is an int and that they want to use in an int context -- but then when it's not even close to an int, they won't notice. Let's call the whole thing off, --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Mon Apr 15 23:20:22 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 15 Apr 2002 18:20:22 -0400 Subject: [Python-Dev] DeprecationWarnings in the regression tests References: <15547.14987.264776.590478@anthem.wooz.org> <200204152121.g3FLLSt02566@pcp742651pcs.reston01.va.comcast.net> <15547.19546.808399.795215@anthem.wooz.org> <200204152210.g3FMAoq02716@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15547.21030.234775.662955@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> Why do you want this? Did you get a particular warning that GvR> bugs you? If not, let's please call a YAGNI on this. Run the regrtest. We're now getting a bunch of warnings in test_coercion and test_descr. I'm happy to ignore them, but I suspect this may end up as a FAQ if 2.3final's "make test" still emits them. -Barry From neal@metaslash.com Mon Apr 15 23:54:37 2002 From: neal@metaslash.com (Neal Norwitz) Date: Mon, 15 Apr 2002 18:54:37 -0400 Subject: [Python-Dev] autoheader warning for WITH_UNIVERSAL_NEWLINES Message-ID: <3CBB5A2D.32265EBF@metaslash.com> When running autoheader, I get the following warning: autoheader: missing template: WITH_UNIVERSAL_NEWLINES Sorry, I have no idea why. Neal From aahz@pythoncraft.com Tue Apr 16 00:03:41 2002 From: aahz@pythoncraft.com (Aahz) Date: Mon, 15 Apr 2002 19:03:41 -0400 Subject: [Python-Dev] float.decode()? In-Reply-To: <20020415145648.A25455@glacier.arctrix.com> References: <20020415145648.A25455@glacier.arctrix.com> Message-ID: <20020415230341.GA23612@panix.com> On Mon, Apr 15, 2002, Neil Schemenauer wrote: > > How hard would it be to write a method like this for Python floats? I > think it could go a long way in helping people understand how floating > point works. Better yet, look at FixedPoint.py or BCD.py on my home page to see how to convert that to a human-readable form. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ What if there were no rhetorical questions? From tim.one@comcast.net Tue Apr 16 00:22:37 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 15 Apr 2002 19:22:37 -0400 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Objects complexobject.c,2.57,2.58 In-Reply-To: <200204152219.g3FMJN302807@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > I presume that you also object against allowing certain "int- > requiring" operations (like use as sequence index) on floats when > the fraction is zero. Not if you *know* the fractional part is 0. If you don't know that for sure, then you're saying sequence indices are fuzzy, subject to change across platforms and releases. I'm acutely aware of this because I spent 10 years using Cray machines where int(6./3.) == 1. That doesn't happen in a 754 world, and 754 also supplies an "exact" flag so you can know for sure that 24./6. is exact and 24./5. is not. The same kind of thing can still happen, though, if "3." is really the result of an fp computation that just happens to round to something a little bigger than 3.0 on the nose; this in turn could be due to a 1-bit difference in the platform exp() implementation for a given input. Over the years, I've tracked down dozens and dozens of x-platform program failures to such tiny platform differences (and, perhaps paradoxically, I spent more time per year doing that at Dragon than at Cray!). > Unified numbers are rapidly becoming less attractive this way... To the extent that they expose platform accidents, they've always been suspect. A fancy hand calculator gets away with it because it's a single platform where all the numerics are under the control of the system (HW and SW) designers, and they can do whatever it takes to avoid visible surprises. They don't care much about speed, either. A pure-SW equivalent would need to stay away from HW fp, or use it in limited known-safe ways. > ... > To me, unified numbers means that there's only one numeric type, and > it always has the same properties. That means that all numbers must > have an imag field, which is zero when it's not a complex number. > > But it seems you want some other indicator that says it's never been > part of a calculation involving complex numbers... If you're to avoid platform accidents, it doesn't matter whether it's ever been complex, what matters is that when you're "downcasting" you *know* whether that's safe, or whether you're just guessing. CL refuses to guess; Scheme tries to make safe guesses possible, but half-heartedly. [on an "exact?" bit] > Unified numbers are becoming even less attractive... It's taken from Scheme: exactness is Scheme's way of trying to allow for guaranteed-safe downcasting (or "down towering") when possible. In practice, though, the Scheme standard says so little about how inexactness propagates that you can't count on it across Scheme implementations. [CL pointers] > Too much to read. :-( Approximate computer arithmetic is a bitch. If you want a system that's easy to define and to use, stay away from HW fp and it's almost easy. Else rounding errors are inescapable, and so have to be dealt with. > But if the end result is that users will write trunc() or round() > calls whenever they have a float value that they believe is an int and > that they want to use in an int context -- but then when it's not even > close to an int, they won't notice. I was wondering when someone would get to that . Implicit in that complaint is that, if we left magical float->int up to "the system", then the system could benevolently decide when a given float was or wasn't "close enough" to an exact int to make no difference. Been there, done that, too. IMO it just increases the complication, and the difficulty of writing correct portable code: it's another layer of hidden magic to trip over. For example, if you decide that 1.9 is "close enough" to 2 that it should be treated as 2 on the nose, but that anything in (1.1, 1.9) is "too far away", then you've effectively just moved the boundaries where the tiniest possible 1-bit rounding accidents lead to radically different behavior. Systems that do stuff like that (APL is an example) then grow global "fuzz" settings so that users can override the system, when, through painful experience, they learn the system *can't* guess reliably. Some hand calculators deal with this by computing to several extra decimal digits internally, and then saying "it's close enough to an int" if and only if the extra-precision hidden result rounds to an exact normal-precision int. That's effective, but perhaps would break down if hand calculators normally did millions (billions, ...) of computations before needing to decide. Forget complex. How is sequence.__getitem__(x) to be defined for a float x (this is the under-the-covers "float"; whether that's a user-visible type distinction isn't material to the question here)? Scheme says that in (list-ref sequence x), x must be an "exact integer", where "integer" includes things like a complex with a 0 imaginary part, but "exact" says you've got to *know* it's a 0 imaginary part, not just an approximation to a 0 imaginary part. That would be sane, except that most Scheme implementations call *all* internal floats "inexact", so in practice it ends up meaning "an internal integer", and all the words allowing for other possiblities just confuse the issue. Common Lisp says "screw that -- if you think this is usable as an int, you convert it to an int yourself: your choice, your responsibility, and here are 4 convenient functions for choosing the rounding you want". Either is preferable to "don't worry, trust me" . From tim.one@comcast.net Tue Apr 16 00:27:52 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 15 Apr 2002 19:27:52 -0400 Subject: [Python-Dev] DeprecationWarnings in the regression tests In-Reply-To: <15547.21030.234775.662955@anthem.wooz.org> Message-ID: [Barry A. Warsaw] > Run the regrtest. We're now getting a bunch of warnings in > test_coercion and test_descr. I'm happy to ignore them, but I suspect > this may end up as a FAQ if 2.3final's "make test" still emits them. It won't. I suppressed a whole bunch of other DeprecationWarnings in the test suite last week, and that's easy to do (albeit tedious). I was lazy, though, and didn't add a call to resetwarnings() in each module where I added a warning filter; indeed, we never call resetwarnings() anywhere now. From guido@python.org Tue Apr 16 01:01:48 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 15 Apr 2002 20:01:48 -0400 Subject: [Python-Dev] autoheader warning for WITH_UNIVERSAL_NEWLINES In-Reply-To: Your message of "Mon, 15 Apr 2002 18:54:37 EDT." <3CBB5A2D.32265EBF@metaslash.com> References: <3CBB5A2D.32265EBF@metaslash.com> Message-ID: <200204160001.g3G01mT03021@pcp742651pcs.reston01.va.comcast.net> > When running autoheader, I get the following warning: > > autoheader: missing template: WITH_UNIVERSAL_NEWLINES > > Sorry, I have no idea why. I see this too. I'm guessing Jack missed some autoconf detail. Martin? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 16 01:10:28 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 15 Apr 2002 20:10:28 -0400 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Objects complexobject.c,2.57,2.58 In-Reply-To: Your message of "Mon, 15 Apr 2002 19:22:37 EDT." References: Message-ID: <200204160010.g3G0ASU03057@pcp742651pcs.reston01.va.comcast.net> [me] > > But if the end result is that users will write trunc() or round() > > calls whenever they have a float value that they believe is an int > > and that they want to use in an int context -- but then when it's > > not even close to an int, they won't notice. [Tim] > I was wondering when someone would get to that . Implicit in > that complaint is that, if we left magical float->int up to "the > system", then the system could benevolently decide when a given > float was or wasn't "close enough" to an exact int to make no > difference. Can I raise a ChannelingError? That was not at all where my thoughts went (I know better than that). If I was going anywhere, it was against unified numbers, period. [...] > Common Lisp says "screw that -- if you think this is usable as an > int, you convert it to an int yourself: your choice, your > responsibility, and here are 4 convenient functions for choosing the > rounding you want". I just hope there's something more refined than round() and trunc(), so that I can easily apply my *own* fuzz. Maybe round(x, n) could return an int when the result is an int (which for round(x, n) is well-defined unless x or n is very large). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 16 01:11:54 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 15 Apr 2002 20:11:54 -0400 Subject: [Python-Dev] DeprecationWarnings in the regression tests In-Reply-To: Your message of "Mon, 15 Apr 2002 19:27:52 EDT." References: Message-ID: <200204160011.g3G0Bsn03073@pcp742651pcs.reston01.va.comcast.net> > I was lazy, though, and didn't add a call to resetwarnings() in each > module where I added a warning filter; indeed, we never call > resetwarnings() anywhere now. You don't need to, as long as the filterwarnings() arguments specify the current file and the specific error message you're suppressing. Calling resetwarnings() should be avoided, because it may enable warnings that were disabled by default or globally on purpose. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Tue Apr 16 01:13:42 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 15 Apr 2002 20:13:42 -0400 Subject: [Python-Dev] DeprecationWarnings in the regression tests In-Reply-To: Message-ID: Damn. warnings.resetwarnings() doesn't really """Reset the list of warnings filters to its default state.""" if by "its default state" one means (as I do ) its initial state upon importing the warnings module. I had the clever idea of calling warnings.resetwarnings() in regrtest.py after running each test, at least so that the damage done by a sloppy overly-general warning suppression added by one test couldn't hurt error reporting in other tests. But an unwanted side effect is that I'm getting brand new "surprise" warnings now, because warnings.py does filterwarnings("ignore", category=OverflowWarning, append=1) as part of its initialization, but resetwarnings nukes that. What should resetwarnings really do? Offhand it looks most reasonable to me to move the module initialization code _processoptions(sys.warnoptions) filterwarnings("ignore", category=OverflowWarning, append=1) into the body of resetwarnings(), then call the latter from the initialization block. Objections? From guido@python.org Tue Apr 16 01:22:00 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 15 Apr 2002 20:22:00 -0400 Subject: [Python-Dev] DeprecationWarnings in the regression tests In-Reply-To: Your message of "Mon, 15 Apr 2002 20:13:42 EDT." References: Message-ID: <200204160022.g3G0M0S09275@pcp742651pcs.reston01.va.comcast.net> > What should resetwarnings really do? Offhand it looks most > reasonable to me to move the module initialization code > > _processoptions(sys.warnoptions) > filterwarnings("ignore", category=OverflowWarning, append=1) > > into the body of resetwarnings(), then call the latter from the > initialization block. > > Objections? If you do that, side effects from _processoptions(), like warnings about incomprehensible -W options, are issued each time. I'd prefer it if you didn't try to call resetwarnings() at all. That's for the main() app only. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Tue Apr 16 01:26:36 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 15 Apr 2002 20:26:36 -0400 Subject: [Python-Dev] DeprecationWarnings in the regression tests In-Reply-To: <200204160022.g3G0M0S09275@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > If you do that, side effects from _processoptions(), like warnings > about incomprehensible -W options, are issued each time. > > I'd prefer it if you didn't try to call resetwarnings() at all. > That's for the main() app only. OK, but the base question still stands: the resetwarnings docstring doesn't match what the code does: is it intended to clear all warning filters (that's what it actually does), or return them to the default state (which is what it says it does, and the state that's there when the module is first imported is about the only way I can read "default"). From greg@cosc.canterbury.ac.nz Tue Apr 16 01:27:38 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 16 Apr 2002 12:27:38 +1200 (NZST) Subject: [Python-Dev] Optik In-Reply-To: <200204151530.g3FFUDe26272@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204160027.MAA01292@s454.cosc.canterbury.ac.nz> > How about calling the module OptionParser I would vote for something like CommandLineParser which more explicitly says what it's about. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim.one@comcast.net Tue Apr 16 01:54:43 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 15 Apr 2002 20:54:43 -0400 Subject: [Python-Dev] DeprecationWarnings in the regression tests In-Reply-To: <200204160034.g3G0YBZ09416@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > You don't need to, as long as the filterwarnings() arguments specify > the current file and the specific error message you're suppressing. Well, that doesn't work: the current file appears to be irrelevant for deprecation warnings, and I'm not sure what "the module" argument should be. For example, in test___all__.py, this works (trial and error) to suppress the statcache warning: warnings.filterwarnings("ignore", ".* statcache .*", DeprecationWarning, 'statcache') But I'm not sure why I'm typing 'statcache' as the last argument. Is this meant to be a module basename, or some flavor of file path? If we don't reset warnings after each test, and the name of the test file is usually irrelevant, then changing the *order* in which we run the tests may turn up new warnings (suppressed in the normal order of running the tests because each filterwarnings call affects every test run after the test that calls it). From mhammond@skippinet.com.au Tue Apr 16 03:52:11 2002 From: mhammond@skippinet.com.au (Mark Hammond) Date: Tue, 16 Apr 2002 12:52:11 +1000 Subject: [Python-Dev] Optik In-Reply-To: <200204160027.MAA01292@s454.cosc.canterbury.ac.nz> Message-ID: [Greg] > > How about calling the module OptionParser > > I would vote for something like CommandLineParser > which more explicitly says what it's about. +1. Also rename the class to CommandLineParser, possibly adding an alias to the existing name to help existing users. % python -c "import this" | grep implicit ly, Mark. From tim.one@comcast.net Tue Apr 16 04:44:00 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 15 Apr 2002 23:44:00 -0400 Subject: [Python-Dev] DeprecationWarnings in the regression tests In-Reply-To: Message-ID: FYI, there are no more warnings when running the test suite on Windows. regrtest *usually* fails in test___all__ when run with -r ("random"), though, due to rlcompleter. It smells like another one of those deals where an expected (on Windows) failing import leaves behind a damaged module object in sys.modules, allowing the next import of it to succeed in another test, but delivering a module object that doesn't have the attributes the latter test is trying to use. So *boom* -- an unexpected AttributeError, and a legitimate error at the time it happens. From barry@zope.com Tue Apr 16 06:08:23 2002 From: barry@zope.com (Barry A. Warsaw) Date: Tue, 16 Apr 2002 01:08:23 -0400 Subject: [Python-Dev] Optik References: <200204151530.g3FFUDe26272@pcp742651pcs.reston01.va.comcast.net> <200204160027.MAA01292@s454.cosc.canterbury.ac.nz> Message-ID: <15547.45511.486798.546891@anthem.wooz.org> >>>>> "GE" == Greg Ewing writes: >> How about calling the module OptionParser GE> I would vote for something like CommandLineParser GE> which more explicitly says what it's about. I don't much mind (and I figure it's largely up to Greg), but please call the class the same thing as the module. If it's CommandLineParser.py, make the class CommandLineParser. -Barry From martin@v.loewis.de Tue Apr 16 06:56:47 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 16 Apr 2002 07:56:47 +0200 Subject: [Python-Dev] autoheader warning for WITH_UNIVERSAL_NEWLINES In-Reply-To: <200204160001.g3G01mT03021@pcp742651pcs.reston01.va.comcast.net> References: <3CBB5A2D.32265EBF@metaslash.com> <200204160001.g3G01mT03021@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > > autoheader: missing template: WITH_UNIVERSAL_NEWLINES > > > > Sorry, I have no idea why. > > I see this too. I'm guessing Jack missed some autoconf detail. > Martin? The "template" is the fragment /* Define if you want to read files with foreign newlines. */ #undef WITH_UNIVERSAL_NEWLINES autoheader generates this from the AC_DEFINE line. If that is not possible because the AC_DEFINE does not provide the docstring, it generates it from acconfig.h. If that is not possible because acconfig.h is not used, it complains (and puts noting into pyconfig.h.in). The fix for this is - AC_DEFINE(WITH_UNIVERSAL_NEWLINES) + AC_DEFINE(WITH_UNIVERSAL_NEWLINES, 1, + [Define if you want to read files with foreign newlines.]) Committed as configure.in 1.311. Regards, Martin P.S. The "1", which is the value this gets if defined, is not strictly necessary. However, explicit is better than implicit. From mwh@python.net Tue Apr 16 09:49:43 2002 From: mwh@python.net (Michael Hudson) Date: 16 Apr 2002 09:49:43 +0100 Subject: [Python-Dev] Inplace multiply by float In-Reply-To: Todd Miller's message of "Mon, 15 Apr 2002 17:54:07 -0400" References: <3CBB4BFF.4050707@stsci.edu> Message-ID: <2mr8lgf2oo.fsf@starship.python.net> Todd Miller writes: > I take this to mean that python does not (currently) support inplace > multiply by floats for new style classes. Is this a bug? A > permanent limitation? Looks like a bug. python-dev is a really good place to lose bug reports . Cheers, M. -- Haha! You had a *really* weak argument! -- Moshe Zadka, comp.lang.python From Jack.Jansen@cwi.nl Tue Apr 16 10:28:32 2002 From: Jack.Jansen@cwi.nl (Jack Jansen) Date: Tue, 16 Apr 2002 11:28:32 +0200 Subject: [Python-Dev] autoheader warning for WITH_UNIVERSAL_NEWLINES In-Reply-To: Message-ID: <51DBD411-511C-11D6-9DE5-0030655234CE@cwi.nl> Martin, how do I find out about all this autoconf magic? I simply used the code for another configure option (WITH_GC, I think) as a template and modified it for my needs. But I'm really groping in the dark with configure. I've have never heard of autoheader, for instance, and haven't a clue that I should have used it (if I should have, and for what:-) On Tuesday, April 16, 2002, at 07:56 , Martin v. Loewis wrote: > Guido van Rossum writes: > >>> autoheader: missing template: WITH_UNIVERSAL_NEWLINES >>> >>> Sorry, I have no idea why. >> >> I see this too. I'm guessing Jack missed some autoconf detail. >> Martin? > > The "template" is the fragment > > /* Define if you want to read files with foreign newlines. */ > #undef WITH_UNIVERSAL_NEWLINES > > autoheader generates this from the AC_DEFINE line. If that is not > possible because the AC_DEFINE does not provide the docstring, it > generates it from acconfig.h. If that is not possible because > acconfig.h is not used, it complains (and puts noting into > pyconfig.h.in). > > The fix for this is > > - AC_DEFINE(WITH_UNIVERSAL_NEWLINES) > + AC_DEFINE(WITH_UNIVERSAL_NEWLINES, 1, > + [Define if you want to read files with foreign newlines.]) > > Committed as configure.in 1.311. > > Regards, > Martin > > P.S. The "1", which is the value this gets if defined, is not strictly > necessary. However, explicit is better than implicit. > -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From guido@python.org Tue Apr 16 13:41:15 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 16 Apr 2002 08:41:15 -0400 Subject: [Python-Dev] autoheader warning for WITH_UNIVERSAL_NEWLINES In-Reply-To: Your message of "Tue, 16 Apr 2002 11:28:32 +0200." <51DBD411-511C-11D6-9DE5-0030655234CE@cwi.nl> References: <51DBD411-511C-11D6-9DE5-0030655234CE@cwi.nl> Message-ID: <200204161241.g3GCfFY20191@pcp742651pcs.reston01.va.comcast.net> > how do I find out about all this autoconf magic? I simply used the code > for another configure option (WITH_GC, I think) as a template and > modified it for my needs. But I'm really groping in the dark with > configure. I've have never heard of autoheader, for instance, and > haven't a clue that I should have used it (if I should have, and for > what:-) I think Martin redid the configure.in stuff just after you were done with your patch but before you checked it in. :-) autoheader is used to create the pyconfig.h.in file. It's run by the Makefile when you say "make autoconf". A common mistake when adding a new configure check or option is to add the desired symbol directly into pyconfig.h.in. The correct solution is currently to use a 3-arg AC_DEFINE call as Martin showed; until last week, it was to add a few lines to acconfig.h (which no longer exists). Docs for autoconf and autoheader are in info form. If you're still a vi(m) user :-), "info autoheader" at the shell command line will tell you more. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Tue Apr 16 15:34:27 2002 From: skip@pobox.com (Skip Montanaro) Date: Tue, 16 Apr 2002 09:34:27 -0500 Subject: [Python-Dev] autoheader warning for WITH_UNIVERSAL_NEWLINES In-Reply-To: <51DBD411-511C-11D6-9DE5-0030655234CE@cwi.nl> References: <51DBD411-511C-11D6-9DE5-0030655234CE@cwi.nl> Message-ID: <15548.13939.654897.445701@12-248-41-177.client.attbi.com> Jack> how do I find out about all this autoconf magic? There are a number of sources for this sort of stuff. One page that looks promising from a Google search is http://www.sunsite.ualberta.ca/Documentation/Gnu/gnu-docs.html It has autoconf and automake docs. Another useful is the "Goat Book", available in dead tree and online formats: http://sources.redhat.com/autobook/ I like that site because it talks about using automake, autoconf and libtool together. Skip From tomalley@eink.com Tue Apr 16 16:30:23 2002 From: tomalley@eink.com (Tim O' Malley) Date: Tue, 16 Apr 2002 11:30:23 -0400 Subject: [Python-Dev] timeoutsocket.py Message-ID: <404B670C5EC4D411873400508B2CDCF2FC5151@mail.eink.com> hola. Regarding timeoutsocket.py: I would be happy to see timeoutsocket.py in the standard library. It's been great that people have found timeoutsocket.py to be useful -- even more so, thrilling that some have found it useful enough to consider inclusion in the standard library. Onto questions... 1. Shall we merge socket.py and timeoutsocket.py? It certainly is possible to merge TimeoutSocket and _socketobject. If the merge was done in a straightforward manner, then every TCP socket would use Python code (instead of C code) for recv() and send() methods. Some might consider this an unacceptable performance hit. It may be possible to "trigger" the creation of a timeoutsocket when the program attempts to set the timeout. (I say "may" because I would need to think about what would happen if someone called makefile() and then tried to set the timeout.) My first inclination: it is "correct" to unify the two objects because that would present the most natural interface. From where the module is today, however, it might take some thought to do it right. 2. If yes to Q1, how are we going to due with the copyright issue? I would be happy to change the copyright over to PSF. There should be some way that users of earlier versions of Python can continue to use timeoutsocket.py, but changing the copyright shouldn't prevent that. 3. If no to Q1, are we going to just add timeoutsocket.py to standard library (Though there the copyright issue stays)? Copyright won't be an issue. TimO -----Original Message----- From: Bernard Yue [mailto:bernie@3captus.com] Sent: Monday, April 15, 2002 4:38 PM To: skip@pobox.com Cc: Guido van Rossum; python-dev@python.org; timo@alum.mit.edu Subject: Re: [Python-Dev] timeoutsocket.py Skip Montanaro wrote: > >> What is the status of integrating timeoutsocket.py into standard > >> library? Are there any body responsible for it currently? If not I > >> would like to give it a shot. > > Guido> Isn't one of the problems that the timeout is global? > > That's the easiest way to use it, but you can specify timeouts on a > per-socket basis. From the module docstring: > > import timeoutsocket > import httplib > H = httplib.HTTP("www.python.org") > H.sock.set_timeout(30) > > The author, Tim O'Malley, is probably the best person to address issues with > this package, so I've added him to the cc list in case he's not a python-dev > subscriber. > > Skip Just have a look at socket.py and timeoutsocket.py (version 1.21, assuming the lastest). Looks like that class _socketobject and class Timeoutsocket can be merged with no conflict. Now it comes the question: 1. Shall we merge socket.py and timeoutsocket.py? 2. If yes to Q1, how are we going to due with the copyright issue? 3. If no to Q1, are we going to just add timeoutsocket.py to standard library (Though there the copyright issue stays)? Bernie -- In Scotland, a new game was invented. It was entitled Gentlemen Only Ladies Forbidden.... and thus the word GOLF entered into the English language. From martin@v.loewis.de Tue Apr 16 16:54:08 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 16 Apr 2002 17:54:08 +0200 Subject: [Python-Dev] autoheader warning for WITH_UNIVERSAL_NEWLINES In-Reply-To: <51DBD411-511C-11D6-9DE5-0030655234CE@cwi.nl> References: <51DBD411-511C-11D6-9DE5-0030655234CE@cwi.nl> Message-ID: Jack Jansen writes: > how do I find out about all this autoconf magic? I simply used the > code for another configure option (WITH_GC, I think) as a template and > modified it for my needs. But I'm really groping in the dark with > configure. I've have never heard of autoheader, for instance, and > haven't a clue that I should have used it (if I should have, and for > what:-) Well, I recommend to read the documentation :-) You need an info viewer, (such as info(1), or Emacs), and there you navigate to autoconf. Most useful is (autoconf)Autoconf Macro Index, which lets you easily find the documentation of each macro. Make sure you read the documentation of each macro before typing it into configure.in. To learn about autoheader, I recommend to read (autoconf)Introduction, in particular (autoconf)Making `configure' Scripts . Regards, Martin From nas@python.ca Tue Apr 16 17:12:27 2002 From: nas@python.ca (Neil Schemenauer) Date: Tue, 16 Apr 2002 09:12:27 -0700 Subject: [Python-Dev] reading info files In-Reply-To: ; from martin@v.loewis.de on Tue, Apr 16, 2002 at 05:54:08PM +0200 References: <51DBD411-511C-11D6-9DE5-0030655234CE@cwi.nl> Message-ID: <20020416091227.A28291@glacier.arctrix.com> Martin v. Loewis wrote: > Well, I recommend to read the documentation :-) You need an info > viewer, (such as info(1), or Emacs), and there you navigate to > autoconf. I recommend pinfo. It's much easier to use than info(1) or the info browser in Emacs (IMHO). I never can remember the bindings for info. Neil From pinard@iro.umontreal.ca Tue Apr 16 18:27:29 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 16 Apr 2002 13:27:29 -0400 Subject: [Python-Dev] Re: reading info files In-Reply-To: <20020416091227.A28291@glacier.arctrix.com> References: <51DBD411-511C-11D6-9DE5-0030655234CE@cwi.nl> <20020416091227.A28291@glacier.arctrix.com> Message-ID: [Neil Schemenauer] > I never can remember the bindings for info. One has to remember the words Prev, Next and Up. These are even written at the top of each Info page. And whenever you see Menu, then Menu as well. Add SPC to move forward, and Quit. One does not really need anything more. You might then enjoy Search and other goodies, if adventurous enough. :-) It's rather hard for me to really believe that one cannot remember these few words. :-) Someone says Info is difficult, others repeat it without having really tried, I guess, and it soon becomes an unfounded rumour. -- François Pinard http://www.iro.umontreal.ca/~pinard From Juergen Hermann" Hi! I had a problem yesterday where I wanted to extend a top-level module by a module in a subpackage. I naively named my module the same as the toplevel module, and peer import bit me. I finally had to rename my module for no good reason to get things working. So my question is: is there an easy way to force a top-level import (adress the top-level namespace), that I overlooked? If not, what do you think of a from __top__ import systemmodule similar to from __future__ ... An alternative syntax would be "import .systemmodule", but that seems obscure. Ciao, J=FCrgen From brian@sweetapp.com Tue Apr 16 23:00:41 2002 From: brian@sweetapp.com (Brian Quinlan) Date: Tue, 16 Apr 2002 15:00:41 -0700 Subject: [Python-Dev] Importing top-level modules In-Reply-To: <16xaZy-18vF4qC@fwd00.sul.t-online.com> Message-ID: <009c01c1e592$2692def0$445d4540@Dell2> Juergen wrote: > I had a problem yesterday where I wanted to extend a top-level module > by a module in a subpackage. I naively named my module the same as the > toplevel module, and peer import bit me. I finally had to rename my > module for no good reason to get things working. What you are trying to do seems likely to cause confusion. But if you really need this, couldn't you just mangle sys.path for a single import and then restore it? It's kind of ugly but it would work. > So my question is: is there an easy way to force a top-level import > (adress the top-level namespace), that I overlooked? > > If not, what do you think of a > > from __top__ import systemmodule > > similar to from __future__ ... I don't like this idea. I'd hate to encourage people to give their subpackages the same name as system modules: from package import string # many lines of code later string.do_something_wierd() # what?!? > An alternative syntax would be "import .systemmodule", but that seems > obscure. That's grotesque. Especially since I think of a leading period as meaning current directory :-) Cheers, Brian From guido@python.org Tue Apr 16 23:37:07 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 16 Apr 2002 18:37:07 -0400 Subject: [Python-Dev] Importing top-level modules In-Reply-To: Your message of "Tue, 16 Apr 2002 23:34:12 +0200." <16xaZy-18vF4qC@fwd00.sul.t-online.com> References: <16xaZy-18vF4qC@fwd00.sul.t-online.com> Message-ID: <200204162237.g3GMb7L27726@pcp742651pcs.reston01.va.comcast.net> > I had a problem yesterday where I wanted to extend a top-level module > by a module in a subpackage. I naively named my module the same as the > toplevel module, and peer import bit me. I finally had to rename my > module for no good reason to get things working. You did the right thing, for a good reason. > So my question is: is there an easy way to force a top-level import > (adress the top-level namespace), that I overlooked? No. > If not, what do you think of a > > from __top__ import systemmodule > > similar to from __future__ ... > > An alternative syntax would be "import .systemmodule", but that seems > obscure. I'm not keen on this. I'd rather deprecate relative imports -- they cause more trouble than they're worth. --Guido van Rossum (home page: http://www.python.org/~guido/) From niemeyer@conectiva.com Tue Apr 16 23:52:58 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Tue, 16 Apr 2002 19:52:58 -0300 Subject: [Python-Dev] Zipfile and tarfile In-Reply-To: <3CB99C8A.9040804@itamarst.org> References: <3CB99C8A.9040804@itamarst.org> Message-ID: <20020416195258.B6738@ibook.distro.conectiva> Hi Itamar! > Well, there's a problem with this - zipfile currently gives no way for users [...] Thank you!! I included your message in the discussion with Lars. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From zilong@trans-cosmos.com.cn Wed Apr 17 03:15:12 2002 From: zilong@trans-cosmos.com.cn (zilong) Date: Wed, 17 Apr 2002 10:15:12 +0800 Subject: [Python-Dev] re Message-ID: <003101c1e5b5$b49cc9e0$730417ac@transcosmos.com.cn> This is a multi-part message in MIME format. ------=_NextPart_000_002E_01C1E5F8.C2A61940 Content-Type: text/plain; charset="gb2312" Content-Transfer-Encoding: quoted-printable thank you i am happy 8-) ------=_NextPart_000_002E_01C1E5F8.C2A61940 Content-Type: text/html; charset="gb2312" Content-Transfer-Encoding: quoted-printable
thank you
 
i am happy
 
8-)
------=_NextPart_000_002E_01C1E5F8.C2A61940-- From Anthony Baxter Wed Apr 17 05:17:08 2002 From: Anthony Baxter (Anthony Baxter) Date: Wed, 17 Apr 2002 14:17:08 +1000 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: Message from Guido van Rossum of "Fri, 12 Apr 2002 13:57:24 -0400." <200204121757.g3CHvOu11530@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204170417.g3H4H8o20585@localhost.localdomain> >>> Guido van Rossum wrote > > Hm. I also want to work with tar and tar.gz files. Is there any > > intend to add a tarfile module to the Python library? > > Dunno, but it's a pretty simple format, so should be easy. Maybe one > of the rookie developers would like to tackle this? If it's wanted, I have a cpio reader class somewhere that could be thrown in. Note that (from memory) it only handles some of the many, many different cpio formats... -- Anthony Baxter It's never too late to have a happy childhood. From guido@python.org Wed Apr 17 13:09:34 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 17 Apr 2002 08:09:34 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: Your message of "Wed, 17 Apr 2002 14:17:08 +1000." <200204170417.g3H4H8o20585@localhost.localdomain> References: <200204170417.g3H4H8o20585@localhost.localdomain> Message-ID: <200204171209.g3HC9YA29287@pcp742651pcs.reston01.va.comcast.net> > If it's wanted, I have a cpio reader class somewhere that could be > thrown in. Note that (from memory) it only handles some of the many, > many different cpio formats... cpio? What's that? An RPM reader, now that would be useful. --Guido van Rossum (home page: http://www.python.org/~guido/) From akuchlin@mems-exchange.org Wed Apr 17 13:21:26 2002 From: akuchlin@mems-exchange.org (akuchlin@mems-exchange.org) Date: Wed, 17 Apr 2002 08:21:26 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <200204171209.g3HC9YA29287@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Wed, Apr 17, 2002 at 08:09:34AM -0400 References: <200204170417.g3H4H8o20585@localhost.localdomain> <200204171209.g3HC9YA29287@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020417082126.A28098@mems-exchange.org> On Wed, Apr 17, 2002 at 08:09:34AM -0400, Guido van Rossum wrote: >> If it's wanted, I have a cpio reader class somewhere that could be >> thrown in. Note that (from memory) it only handles some of the many, >> many different cpio formats... > >cpio? What's that? An RPM reader, now that would be useful. cpio is a program like tar, that reads tar files in addition to its own format. Unfortunately there are a bunch of different cpio formats; see the GNU manpage for cpio for a list. To answer your third sentence, note that the RPM tools include rpm2cpio. Also, Red Hat's rpm module likely includes functions for reading RPM file. --amk (www.amk.ca) Cleverness: A predisposition to irritate excessively. -- Peter Greenaway, _Rosa: The Death of a Composer_ From Juergen Hermann" Message-ID: <16xtB0-1Xh1E0C@fwd09.sul.t-online.com> On Tue, 16 Apr 2002 18:37:07 -0400, Guido van Rossum wrote: >I'm not keen on this. I'd rather deprecate relative imports -- they >cause more trouble than they're worth. That would be fine, too. Ciao, J=FCrgen From martin@v.loewis.de Wed Apr 17 19:18:45 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 17 Apr 2002 20:18:45 +0200 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <20020417082126.A28098@mems-exchange.org> References: <200204170417.g3H4H8o20585@localhost.localdomain> <200204171209.g3HC9YA29287@pcp742651pcs.reston01.va.comcast.net> <20020417082126.A28098@mems-exchange.org> Message-ID: akuchlin@mems-exchange.org writes: > cpio is a program like tar, that reads tar files in addition to its > own format. Unfortunately there are a bunch of different cpio > formats; see the GNU manpage for cpio for a list. The recent Posix release deprecates the cpio(1) utility, and replaces it with the pax(1) utility. By default, pax writes the ustar format, but with enhanced features taken from cpio. In addition, it fixes bugs in both formats, e.g. by introducing the notion of character sets and Unicode for file names in tar files. As for accessing rpm: Redhat's rpm module does that, which is part of the standard RPM distribution. Of course, it is a C extension, using the RPM library. Regards, Martin From geoff-pdev@gerrietts.net Wed Apr 17 20:34:15 2002 From: geoff-pdev@gerrietts.net (Geoff Gerrietts) Date: Wed, 17 Apr 2002 12:34:15 -0700 Subject: [Python-Dev] core out of select.so Message-ID: <20020417193415.GA28494@isis.gerrietts.net> I'm currently able to pretty reliably produce a core file using Zope 2.5.0 and Python 2.1.3 on a box based on RedHat 6.2. The core is produced due to a segfault, from in select.so. I can look at the backtrace, and I've done some preliminary inspection of the active code object, the call out to select.so, etc. I haven't learned much as a consequence of that, unfortunately: the args passed to the C function don't appear to be a valid python object (can't access the type object), though the function seems okay; and I don't appear to have a filename for my code object (just /). My backtraces appear to vary in quality (not sure why) -- sometimes I get all the symbols I expect, sometimes I get no symbols at all. The best traceback I have at the moment (though I I'm writing now in the hopes that some of the regulars here have some standard recipes for extracting useful information from the backtraces that I might get something approaching a reconstruction of the python code that's being executed, and the arguments that are being supplied. If nobody's got any hints, that's fine, I'm still working on it -- but if you do, I'd appreciate it. Thanks, --G. -- Geoff Gerrietts "A man can't be too careful in the choice of his enemies." --Oscar Wilde From akuchlin@mems-exchange.org Wed Apr 17 21:00:54 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Wed, 17 Apr 2002 16:00:54 -0400 Subject: [Python-Dev] core out of select.so In-Reply-To: <20020417193415.GA28494@isis.gerrietts.net> References: <20020417193415.GA28494@isis.gerrietts.net> Message-ID: <20020417200054.GA30667@ute.mems-exchange.org> On Wed, Apr 17, 2002 at 12:34:15PM -0700, Geoff Gerrietts wrote: >My backtraces appear to vary in quality (not sure why) -- sometimes I >get all the symbols I expect, sometimes I get no symbols at all. The >best traceback I have at the moment (though I Please create a bug report on SourceForge for it (or send a traceback or two to me privately, and I can file a bug report). If the tracebacks show that the segfault is occurring at different places in the C code, it's possible that some extension has a refcounting bug or is overwriting malloc'ed memory. The actual crash would then come later in different code that isn't actually buggy. --amk (www.amk.ca) The cheeseboard is the world, and the pieces the phenomena of the universe, as my old friend Huxley used to say. Cheeseboard? -- The Doctor, in "Logopolis" From geoff-pdev@gerrietts.net Wed Apr 17 21:02:12 2002 From: geoff-pdev@gerrietts.net (Geoff Gerrietts) Date: Wed, 17 Apr 2002 13:02:12 -0700 Subject: [Python-Dev] core out of select.so In-Reply-To: <20020417200054.GA30667@ute.mems-exchange.org> References: <20020417193415.GA28494@isis.gerrietts.net> <20020417200054.GA30667@ute.mems-exchange.org> Message-ID: <20020417200212.GB28494@isis.gerrietts.net> Quoting Andrew Kuchling (akuchlin@mems-exchange.org): > On Wed, Apr 17, 2002 at 12:34:15PM -0700, Geoff Gerrietts wrote: > >My backtraces appear to vary in quality (not sure why) -- sometimes I > >get all the symbols I expect, sometimes I get no symbols at all. The > >best traceback I have at the moment (though I > > Please create a bug report on SourceForge for it (or send a traceback > or two to me privately, and I can file a bug report). Has always been my intent, I just want to do my homework, first, and see if I can't turn it into something reproducible that doesn't require mirroring my exact installation and Zope database. > If the tracebacks show that the segfault is occurring at different > places in the C code, it's possible that some extension has a > refcounting bug or is overwriting malloc'ed memory. The actual crash > would then come later in different code that isn't actually buggy. The tracebacks appear to be pointing to the same memory addresses. I'm generating a bunch, and I've gone back and rebuilt python with Py_DEBUG turned on to see what I can find out there. -- Geoff Gerrietts "Whenever people agree with me I always feel I must be wrong." --Oscar Wilde From guido@python.org Wed Apr 17 21:11:32 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 17 Apr 2002 16:11:32 -0400 Subject: [Python-Dev] core out of select.so In-Reply-To: Your message of "Wed, 17 Apr 2002 16:00:54 EDT." <20020417200054.GA30667@ute.mems-exchange.org> References: <20020417193415.GA28494@isis.gerrietts.net> <20020417200054.GA30667@ute.mems-exchange.org> Message-ID: <200204172011.g3HKBWj30924@odiug.zope.com> > On Wed, Apr 17, 2002 at 12:34:15PM -0700, Geoff Gerrietts wrote: > >My backtraces appear to vary in quality (not sure why) -- sometimes I > >get all the symbols I expect, sometimes I get no symbols at all. The > >best traceback I have at the moment (though I > > Please create a bug report on SourceForge for it (or send a traceback > or two to me privately, and I can file a bug report). > > If the tracebacks show that the segfault is occurring at different > places in the C code, it's possible that some extension has a > refcounting bug or is overwriting malloc'ed memory. The actual crash > would then come later in different code that isn't actually buggy. > > --amk Since he mentioned select.so, I'm curious if he might be overflowing the thread stack. That might cause a corrupt stack. There's no way to get a reliable C trace once the stack has been corrupted. You need more info before it makes sense to submit this to SF. --Guido van Rossum (home page: http://www.python.org/~guido/) From geoff-pdev@gerrietts.net Wed Apr 17 21:24:43 2002 From: geoff-pdev@gerrietts.net (Geoff Gerrietts) Date: Wed, 17 Apr 2002 13:24:43 -0700 Subject: [Python-Dev] core out of select.so In-Reply-To: <200204172011.g3HKBWj30924@odiug.zope.com> References: <20020417193415.GA28494@isis.gerrietts.net> <20020417200054.GA30667@ute.mems-exchange.org> <200204172011.g3HKBWj30924@odiug.zope.com> Message-ID: <20020417202443.GC28494@isis.gerrietts.net> Quoting Guido van Rossum (guido@python.org): > Since he mentioned select.so, I'm curious if he might be overflowing > the thread stack. That might cause a corrupt stack. There's no way > to get a reliable C trace once the stack has been corrupted. You need > more info before it makes sense to submit this to SF. I'll agree that it's certainly plausible that I'm running into resource limits. The stack isn't especially deep (16 frames including frame 0), and doesn't appear at first glance to be terribly full, but the ZODB database is pretty large (180MB); it wouldn't surprise me too much if we were encountering resource limits. But color me ignorant if you wish; I have no idea how to go about verifying or testing these conditions, or even how the C stack is different from the thread stack (if it is in fact different). What kind of more information can I dig around and provide you? -- Geoff Gerrietts "Punctuality is the virtue of the bored." --Evelyn Waugh From Jack.Jansen@oratrix.com Wed Apr 17 22:14:24 2002 From: Jack.Jansen@oratrix.com (Jack Jansen) Date: Wed, 17 Apr 2002 23:14:24 +0200 Subject: [Python-Dev] Re: reading info files In-Reply-To: Message-ID: <180D5CE8-5248-11D6-92FE-003065517236@oratrix.com> On dinsdag, april 16, 2002, at 07:27 , Fran=E7ois Pinard wrote: > One has to remember the words Prev, Next and Up. These are=20 > even written at > the top of each Info page. And whenever you see Menu, then=20 > Menu as well. > Add SPC to move forward, and Quit. One does not really need=20 > anything more. > You might then enjoy Search and other goodies, if adventurous=20 > enough. :-) The problem I always have with info files is the navigation if=20 you want to do more than prev/next/up. Plus the often deeply=20 nested structure that you can't break out of (which is why you=20 want more navigation commands). A book is nice: TOC, chapters,=20 index. A good manpage is nice: a 1-3 page document on a single=20 command. Info with its deep structure that it won't reveal to=20 you is cumbersome. And you can't even print it to read it=20 offline (at least: I always end up with an unstructured mess of=20 ASCII lines). -- - Jack Jansen =20 http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution --=20 Emma Goldman - From martin@v.loewis.de Wed Apr 17 23:22:06 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 18 Apr 2002 00:22:06 +0200 Subject: [Python-Dev] core out of select.so In-Reply-To: <20020417202443.GC28494@isis.gerrietts.net> References: <20020417193415.GA28494@isis.gerrietts.net> <20020417200054.GA30667@ute.mems-exchange.org> <200204172011.g3HKBWj30924@odiug.zope.com> <20020417202443.GC28494@isis.gerrietts.net> Message-ID: Geoff Gerrietts writes: > But color me ignorant if you wish; I have no idea how to go about > verifying or testing these conditions, or even how the C stack is > different from the thread stack (if it is in fact different). What > kind of more information can I dig around and provide you? I agree with Andrew: Post what you have somewhere, preferably on SF. It may be that nobody can offer a clue; we'll have to see. Regards, Martin From martin@v.loewis.de Wed Apr 17 23:23:33 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 18 Apr 2002 00:23:33 +0200 Subject: [Python-Dev] Re: reading info files In-Reply-To: <180D5CE8-5248-11D6-92FE-003065517236@oratrix.com> References: <180D5CE8-5248-11D6-92FE-003065517236@oratrix.com> Message-ID: Jack Jansen writes: > The problem I always have with info files is the navigation if you > want to do more than prev/next/up. Plus the often deeply nested > structure that you can't break out of (which is why you want more > navigation commands). A book is nice: TOC, chapters, index. A good > manpage is nice: a 1-3 page document on a single command. Info with > its deep structure that it won't reveal to you is cumbersome. And you > can't even print it to read it offline (at least: I always end up with > an unstructured mess of ASCII lines). You don't print the info file. Instead, you use the texinfo file to convert it to TeX, then print that. For GNU software, you can even order printed manuals from the FSF. Regards, Martin From aahz@pythoncraft.com Wed Apr 17 23:56:21 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 17 Apr 2002 18:56:21 -0400 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: References: <200204170417.g3H4H8o20585@localhost.localdomain> <200204171209.g3HC9YA29287@pcp742651pcs.reston01.va.comcast.net> <20020417082126.A28098@mems-exchange.org> Message-ID: <20020417225621.GA12027@panix.com> On Wed, Apr 17, 2002, Martin v. Loewis wrote: > > The recent Posix release deprecates the cpio(1) utility, and replaces > it with the pax(1) utility. By default, pax writes the ustar format, > but with enhanced features taken from cpio. In addition, it fixes bugs > in both formats, e.g. by introducing the notion of character sets and > Unicode for file names in tar files. Side note for those unfamiliar with this bit of Windows lore: all versions of Windows claiming POSIX compatibility ship the pax utility, and thus can read and write tar files. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ What if there were no rhetorical questions? From geoff-pdev@gerrietts.net Thu Apr 18 00:09:30 2002 From: geoff-pdev@gerrietts.net (Geoff Gerrietts) Date: Wed, 17 Apr 2002 16:09:30 -0700 Subject: [Python-Dev] core out of select.so In-Reply-To: References: <20020417193415.GA28494@isis.gerrietts.net> <20020417200054.GA30667@ute.mems-exchange.org> <200204172011.g3HKBWj30924@odiug.zope.com> <20020417202443.GC28494@isis.gerrietts.net> Message-ID: <20020417230930.GG28494@isis.gerrietts.net> Quoting Martin v. Loewis (martin@v.loewis.de): > Geoff Gerrietts writes: > > > But color me ignorant if you wish; I have no idea how to go about > > verifying or testing these conditions, or even how the C stack is > > different from the thread stack (if it is in fact different). What > > kind of more information can I dig around and provide you? > > I agree with Andrew: Post what you have somewhere, preferably on > SF. It may be that nobody can offer a clue; we'll have to see. I opened the bug and included a represntative backtrace. I'm not sure if I can attach an additional one; I suppose a tarfile or zipfile would have been a sensible alternative. The tracebacks are essentially identical anyway, as the report says. http://sourceforge.net/tracker/index.php?func=detail&aid=545410&group_id=5470&atid=105470 Thanks, --G. -- Geoff Gerrietts "Me and my homies, we tag O.D.." --Unknown grafitti artist at a party From Anthony Baxter Thu Apr 18 03:17:05 2002 From: Anthony Baxter (Anthony Baxter) Date: Thu, 18 Apr 2002 12:17:05 +1000 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: Message from Guido van Rossum of "Wed, 17 Apr 2002 08:09:34 -0400." <200204171209.g3HC9YA29287@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204180217.g3I2H6r07020@localhost.localdomain> >>> Guido van Rossum wrote > > If it's wanted, I have a cpio reader class somewhere that could be > > thrown in. Note that (from memory) it only handles some of the many, > > many different cpio formats... > > cpio? What's that? An RPM reader, now that would be useful. It's another type of file archiver. Depending on who you ask, it's superior to tar because it correctly handles hard links and the like. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From anthony@interlink.com.au Thu Apr 18 03:18:31 2002 From: anthony@interlink.com.au (Anthony Baxter) Date: Thu, 18 Apr 2002 12:18:31 +1000 Subject: [Python-Dev] string.strip and friends for 2.2.2... Message-ID: <200204180218.g3I2IVW07043@localhost.localdomain> I'd like to add the optional argument support for strip and friends to 2.2.2. It solves a real problem, and is useful for people going forward (in the same way as the not-really-bool() code). Opinions? Anthony -- Anthony Baxter It's never too late to have a happy childhood. From guido@python.org Thu Apr 18 05:09:00 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 18 Apr 2002 00:09:00 -0400 Subject: [Python-Dev] string.strip and friends for 2.2.2... In-Reply-To: Your message of "Thu, 18 Apr 2002 12:18:31 +1000." <200204180218.g3I2IVW07043@localhost.localdomain> References: <200204180218.g3I2IVW07043@localhost.localdomain> Message-ID: <200204180409.g3I490j06124@pcp742651pcs.reston01.va.comcast.net> > I'd like to add the optional argument support for strip and friends to > 2.2.2. It solves a real problem, and is useful for people going forward > (in the same way as the not-really-bool() code). Go ahead. Maybe Walter wants to help? (Walter: there's no obligation, and I understand if you prefer not to learn about CVS branches just yet. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From Anthony Baxter Thu Apr 18 06:30:46 2002 From: Anthony Baxter (Anthony Baxter) Date: Thu, 18 Apr 2002 15:30:46 +1000 Subject: [Python-Dev] string.strip and friends for 2.2.2... In-Reply-To: Message from Guido van Rossum of "Thu, 18 Apr 2002 00:09:00 -0400." <200204180409.g3I490j06124@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204180530.g3I5Uk017633@localhost.localdomain> > Go ahead. Maybe Walter wants to help? (Walter: there's no > obligation, and I understand if you prefer not to learn about CVS > branches just yet. :-) It's in. If people are interested, I can post my little auto-cvs-branch-patch- easier somewhere. It's pretty simple. I run a CVS commit email through it, and it generates a list of commands to run. Seems to work ok now... I also put a NEWS message that specifically mentions the rstrip('L') use case, to make people's life easier. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From duncan@rcp.co.uk Thu Apr 18 09:39:12 2002 From: duncan@rcp.co.uk (Duncan Booth) Date: Thu, 18 Apr 2002 09:39:12 +0100 Subject: [Python-Dev] Adding Optik to 2.3 standard library References: <200204170417.g3H4H8o20585@localhost.localdomain> <200204171209.g3HC9YA29287@pcp742651pcs.reston01.va.comcast.net> <20020417082126.A28098@mems-exchange.org> <20020417225621.GA12027@panix.com> Message-ID: <08391204600885@aluminium.rcp.co.uk> On 17 Apr 2002, Aahz wrote: > Side note for those unfamiliar with this bit of Windows lore: all > versions of Windows claiming POSIX compatibility ship the pax utility, > and thus can read and write tar files. Not only that, but if you copy/rename the Windows pax.exe program to tar.exe or cpio.exe it changes its command line options to match. So all recent versions of Windows ship with tar and cpio included (sort of). -- Duncan Booth duncan@rcp.co.uk int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3" "\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure? From thomas.heller@ion-tof.com Thu Apr 18 09:49:41 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 18 Apr 2002 10:49:41 +0200 Subject: [Python-Dev] Adding Optik to 2.3 standard library References: <200204170417.g3H4H8o20585@localhost.localdomain> <200204171209.g3HC9YA29287@pcp742651pcs.reston01.va.comcast.net> <20020417082126.A28098@mems-exchange.org> <20020417225621.GA12027@panix.com> <08391204600885@aluminium.rcp.co.uk> Message-ID: <177f01c1e6b5$fb023720$475afea9@thomasnotebook> From: "Duncan Booth" > On 17 Apr 2002, Aahz wrote: > > > Side note for those unfamiliar with this bit of Windows lore: all > > versions of Windows claiming POSIX compatibility ship the pax utility, > > and thus can read and write tar files. > > Not only that, but if you copy/rename the Windows pax.exe program to > tar.exe or cpio.exe it changes its command line options to match. So all > recent versions of Windows ship with tar and cpio included (sort of). > I didn't know this ;-) On the other hand, I'd prefer to do the packing/unpacking of zip or tar files in Python. I've been hurt more than once by incompatible versions of zip.exe on windows. Thomas From thomas.heller@ion-tof.com Thu Apr 18 13:19:57 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 18 Apr 2002 14:19:57 +0200 Subject: [Python-Dev] getter/setter function signatures Message-ID: <182e01c1e6d3$5ac36f90$475afea9@thomasnotebook> The signature for the getter/setter functions in descrobj.h is typedef PyObject *(*getter)(PyObject *, void *); typedef int (*setter)(PyObject *, PyObject *, void *); At least descrobject.c, cPickle.c and funcobject.c defines PyGetSetDef arrays which use function signatures where the last void* parameter is missing. While obviously this does no harm (so far), it is surely a bug (or am I missing something)? ----- Related: I am always wondering, why the compiler (MSVC 6 in my case) gives a warning when you use these signatures for getter/setter functions PyObject *get(wrapperobject *, void *) int set(wrapperobject *, PyObject *, void *) (warning C4028: formal parameter 1 different from declaration), while if I use these signatures PyObject *get(PyObject *) int set(PyObject *, PyObject *) the compilation gives no warning at all. Another MSVC glitch? Thomas From martin@v.loewis.de Thu Apr 18 13:47:27 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 18 Apr 2002 14:47:27 +0200 Subject: [Python-Dev] getter/setter function signatures In-Reply-To: <182e01c1e6d3$5ac36f90$475afea9@thomasnotebook> References: <182e01c1e6d3$5ac36f90$475afea9@thomasnotebook> Message-ID: "Thomas Heller" writes: > At least descrobject.c, cPickle.c and funcobject.c defines PyGetSetDef > arrays which use function signatures where the last void* parameter is > missing. While obviously this does no harm (so far), it is surely a bug > (or am I missing something)? In the sense of strict C, this is a bug - the function is called with the closure present, so it must also be declared with the closure. Most C implementations support calls where the caller provides more arguments than the callee consumes, for compatibility with K&R C, so this is not a problem. Still, it should be changed, IMO. > Related: I am always wondering, why the compiler (MSVC 6 in my case) > gives a warning when you use these signatures for getter/setter functions > > PyObject *get(wrapperobject *, void *) > int set(wrapperobject *, PyObject *, void *) > > (warning C4028: formal parameter 1 different from declaration), > while if I use these signatures > > PyObject *get(PyObject *) > int set(PyObject *, PyObject *) > > the compilation gives no warning at all. Another MSVC glitch? MSVC is right when giving the warnings about the first set of signatures: You cannot convert PyObject*(*)(wrapperobject*,void*) to PyObject*(*)(PyObject*,void*). Apparently, it sees no problem with the conversion of PyObject*(*)(PyObject*) to PyObject*(*)(PyObject*,void*), which, again, could be for compatibility with K&R C (function pointer type can be converted if the argument types of one function are an initial sequence of the argument types of the other). Just see whether compiling in strict C mode changes anything. Regards, Martin From thomas.heller@ion-tof.com Thu Apr 18 14:52:01 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 18 Apr 2002 15:52:01 +0200 Subject: [Python-Dev] getter/setter function signatures References: <182e01c1e6d3$5ac36f90$475afea9@thomasnotebook> Message-ID: <185b01c1e6e0$374d2530$475afea9@thomasnotebook> As an extension writer, this bugs me [1]. Looking into Python's sources, I find mostly code like this: static PyObject * func_get_code(PyFunctionObject *op) { .... } static PyGetSetDef func_getsetlist[] = { {"func_code", (getter)func_get_code, (setter)func_set_code}, .... {NULL} /* Sentinel */ }; in other words, casting like hell to the correct function type. Should a style like this should be preferred: static PyObject * func_get_code(PyObject *py) { PyFunctionObject *op = (PyFunctionObject *)py; .... } static PyGetSetDef func_getsetlist[] = { {"func_code", func_get_code, func_set_code}, .... {NULL} /* Sentinel */ }; in other words, move the cast where you are more sure about it's correctness, and give the compiler a change for meaningfull warnings. Thomas [1] Actually it's getting better since the documentation has improved a lot, before that I was mainly looking into the source or for examples. From guido@python.org Thu Apr 18 15:02:51 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 18 Apr 2002 10:02:51 -0400 Subject: [Python-Dev] getter/setter function signatures In-Reply-To: Your message of "Thu, 18 Apr 2002 15:52:01 +0200." <185b01c1e6e0$374d2530$475afea9@thomasnotebook> References: <182e01c1e6d3$5ac36f90$475afea9@thomasnotebook> <185b01c1e6e0$374d2530$475afea9@thomasnotebook> Message-ID: <200204181402.g3IE2pH32021@odiug.zope.com> > Looking into Python's sources, I find mostly code like this: > > static PyObject * > func_get_code(PyFunctionObject *op) > { > .... > } > > static PyGetSetDef func_getsetlist[] = { > {"func_code", (getter)func_get_code, (setter)func_set_code}, > .... > {NULL} /* Sentinel */ > }; > > > > in other words, casting like hell to the correct function type. > > Should a style like this should be preferred: > > static PyObject * > func_get_code(PyObject *py) > { > PyFunctionObject *op = (PyFunctionObject *)py; > .... > } > > static PyGetSetDef func_getsetlist[] = { > {"func_code", func_get_code, func_set_code}, > .... > {NULL} /* Sentinel */ > }; > > > in other words, move the cast where you are more sure about it's correctness, > and give the compiler a change for meaningfull warnings. I find that using the first form generally you end up using fewer casts, and that's my preference (because casts obscure the code). This is one area where I wish we could use C++, so e.g. PyFunctionObject would be a subclass of PyObject... :-( --Guido van Rossum (home page: http://www.python.org/~guido/) From walter@livinglogic.de Thu Apr 18 16:06:26 2002 From: walter@livinglogic.de (=?iso-8859-1?Q?Walter_D=F6rwald?=) Date: Thu, 18 Apr 2002 17:06:26 +0200 (CEST) Subject: [Python-Dev] (no subject) Message-ID: <1921.213.167.160.151.1019142386.squirrel@isar.livinglogic.de> Guido van Rossum wrote: > > I'd like to add the optional argument support for strip and friends to > > 2.2.2. It solves a real problem, and is useful for people going forward > > (in the same way as the not-really-bool() code). > > Go ahead. Maybe Walter wants to help? (Walter: there's no > obligation, and I understand if you prefer not to learn about CVS > branches just yet. :-) I have not problems with branches, but I guess that those that do backports regularily have scripts that simplify the job. What's still missing is the version for unicode. Note that my patch http://www.python.org/sf/424606 works a little different: Here the argument is a complete word that should be stripped, not a collection of characters, i.e. your version: "oofoo".strip("foo") => "" my version: "oofoo".strip("foo") => "oo" -- Walter Dörwald · LivingLogic AG, Bayreuth/Germany · www.livinglogic.de From fdrake@acm.org Thu Apr 18 16:20:17 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 18 Apr 2002 11:20:17 -0400 Subject: [Python-Dev] (no subject) In-Reply-To: <1921.213.167.160.151.1019142386.squirrel@isar.livinglogic.de> References: <1921.213.167.160.151.1019142386.squirrel@isar.livinglogic.de> Message-ID: <15550.58417.127070.60711@grendel.zope.com> =?iso-8859-1?Q?Walter_D=F6rwald?= writes: > your version: "oofoo".strip("foo") => "" > my version: "oofoo".strip("foo") => "oo" It sounds like yours is more like Perl's chomp(), with an option about what to chomp off. Not a bad idea, not sure I'd ever use more than: line = line.chomp('\n') -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From skip@pobox.com Thu Apr 18 16:23:55 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 18 Apr 2002 10:23:55 -0500 Subject: [Python-Dev] idle vs. idlefork? Message-ID: <15550.58635.194428.577110@12-248-41-177.client.attbi.com> Just curious... I noticed a couple idle checkins in the past few days. Are these changes going into idlefork? Is anything flowing back from idlefork to idle? Skip From aleax@aleax.it Thu Apr 18 16:24:04 2002 From: aleax@aleax.it (Alex Martelli) Date: Thu, 18 Apr 2002 17:24:04 +0200 Subject: [Python-Dev] (no subject) In-Reply-To: <1921.213.167.160.151.1019142386.squirrel@isar.livinglogic.de> References: <1921.213.167.160.151.1019142386.squirrel@isar.livinglogic.de> Message-ID: On Thursday 18 April 2002 05:06 pm, Walter Dörwald wrote: ... > What's still missing is the version for unicode. Note that my patch > http://www.python.org/sf/424606 works a little different: Here the > argument is a complete word that should be stripped, not a collection > of characters, i.e. > your version: "oofoo".strip("foo") => "" > my version: "oofoo".strip("foo") => "oo" It seems to me that Guido's version is easier to teach, and more useful. Easier to teach, because one can present this as the argument's "default value" being string.whitespace; more useful because I more often have strings to clean up that may end with arbitrary sequences of "junk" characters I want to remove, rather than with a potential specific "junk" word (and the latter case may be easier to handle with an .endswith test). Admittedly the specific use case which Guido recently mentioned (cleaning up, if present, the trailing 'L' from a number's repr) is handled just as well in either way, since it's only one character. But say e.g. I receive datagrams that may or may not end with trailing \r , \n, or \r\n -- .rstrip("\r\n") in Guido's version immediately solves my problem. I can't think of a similarly generic use-case for yours (perhaps a failure of imagination on my part). Alex From skip@pobox.com Thu Apr 18 16:26:17 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 18 Apr 2002 10:26:17 -0500 Subject: [Python-Dev] string.strip and friends for 2.2.2... In-Reply-To: <200204180530.g3I5Uk017633@localhost.localdomain> References: <200204180409.g3I490j06124@pcp742651pcs.reston01.va.comcast.net> <200204180530.g3I5Uk017633@localhost.localdomain> Message-ID: <15550.58777.684501.886966@12-248-41-177.client.attbi.com> Anthony> If people are interested, I can post my little Anthony> auto-cvs-branch-patch- easier somewhere. It's pretty simple. I Anthony> run a CVS commit email through it, and it generates a list of Anthony> commands to run. Seems to work ok now... I wondered why you were always typing "gvanrossum" instead of "Guido" in your checkin messages. :-) Skip From guido@python.org Thu Apr 18 16:51:16 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 18 Apr 2002 11:51:16 -0400 Subject: [Python-Dev] (no subject) In-Reply-To: Your message of "Thu, 18 Apr 2002 17:06:26 +0200." <1921.213.167.160.151.1019142386.squirrel@isar.livinglogic.de> References: <1921.213.167.160.151.1019142386.squirrel@isar.livinglogic.de> Message-ID: <200204181551.g3IFpHR32491@odiug.zope.com> > What's still missing is the version for unicode. Note that my patch > http://www.python.org/sf/424606 works a little different: Here the > argument is a complete word that should be stripped, not a collection > of characters, i.e. > your version: "oofoo".strip("foo") => "" > my version: "oofoo".strip("foo") => "oo" Aargh! I prefer my version -- it's more similar to the default usage (which boils down to something like s.strip(" \t\n\r\f")) and more practical (e.g. stripping trailing punctuation). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Apr 18 16:52:07 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 18 Apr 2002 11:52:07 -0400 Subject: [Python-Dev] idle vs. idlefork? In-Reply-To: Your message of "Thu, 18 Apr 2002 10:23:55 CDT." <15550.58635.194428.577110@12-248-41-177.client.attbi.com> References: <15550.58635.194428.577110@12-248-41-177.client.attbi.com> Message-ID: <200204181552.g3IFq7q32506@odiug.zope.com> > Just curious... I noticed a couple idle checkins in the past few days. Are > these changes going into idlefork? Is anything flowing back from idlefork > to idle? Not systematically. That will be one heck of a merge, at some point. I'm interesting to do that, though, when it's ready. --Guido van Rossum (home page: http://www.python.org/~guido/) From SBrunning@trisystems.co.uk Thu Apr 18 17:09:14 2002 From: SBrunning@trisystems.co.uk (Simon Brunning) Date: Thu, 18 Apr 2002 17:09:14 +0100 Subject: [Python-Dev] (no subject) Message-ID: <31575A892FF6D1118F5800600846864DCBCFF3@intrepid> > From: Guido van Rossum [SMTP:guido@python.org] > > What's still missing is the version for unicode. Note that my patch > > http://www.python.org/sf/424606 works a little different: Here the > > argument is a complete word that should be stripped, not a collection > > of characters, i.e. > > your version: "oofoo".strip("foo") => "" > > my version: "oofoo".strip("foo") => "oo" > > Aargh! I prefer my version -- it's more similar to the default usage > (which boils down to something like s.strip(" \t\n\r\f")) and more > practical (e.g. stripping trailing punctuation). Me too. But what about: "oofoo".strip("foo") => "" "oofoo".strip(["foo", "bar"]) => "oo" I.e., strip could accept *any* sequence (rather than just a string) as its optional parameter, and strip all occurrences of any item in that sequence. More flexible, but I can't think of a good use case, to be honest. Cheers, Simon Brunning TriSystems Ltd. sbrunning@trisystems.co.uk ----------------------------------------------------------------------- The information in this email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this email by anyone else is unauthorised. If you are not the intended recipient, any disclosure, copying, distribution, or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. TriSystems Ltd. cannot accept liability for statements made which are clearly the senders own. From trentm@ActiveState.com Thu Apr 18 17:11:46 2002 From: trentm@ActiveState.com (Trent Mick) Date: Thu, 18 Apr 2002 09:11:46 -0700 Subject: [Python-Dev] string.strip and friends for 2.2.2... In-Reply-To: <200204180530.g3I5Uk017633@localhost.localdomain>; from anthony@interlink.com.au on Thu, Apr 18, 2002 at 03:30:46PM +1000 References: <200204180530.g3I5Uk017633@localhost.localdomain> Message-ID: <20020418091146.I28973@ActiveState.com> [Anthony Baxter wrote] > > > > Go ahead. Maybe Walter wants to help? (Walter: there's no > > obligation, and I understand if you prefer not to learn about CVS > > branches just yet. :-) > > It's in. If people are interested, I can post my little auto-cvs-branch-patch- > easier somewhere. It's pretty simple. I run a CVS commit email through it, and > it generates a list of commands to run. Seems to work ok now... Seems like a candidate for Tools/... Trent -- Trent Mick TrentM@ActiveState.com From Kofi Grey" FROM: GREY KOFI ADMIN.SECRETARY STANDARD SECURITIES. ACCRA, GHANA. DEAR SIR, I do hope this letter will not come to you as a surprise. It was borne out of my desire to share a mutual business relationship with you . My name is GREY KOFI. A 47 years Ghanaian national, married with a wife and four children. I work as an administrative secretary in Standard Securities and Services Limited in Accra-Ghana. I got the information concerning you from the Ghana Chambers of Commerce and Industries after due consultation with my opinion adviser, I decided to contact you believing that by the grace of God, you will accept to be my partner in this business. I joined the services of this company in 1991 as office assistant and have been working with this company for nine years, within this period, I have worked with states and government functionaries who have been using Standard Securities to move huge sums of money U.S Dollars, Pounds Sterling, Foreign Tiakh(Cash)to their foreign partners. They bring in these consignments of money cash and secretly declare the contents as jewelry, Gold, Diamond, Precious stones, Family Treasure, Documents, etc. General Sanni Abacha of Nigeria(dead), Mobutu Sese Sekou of zaire (dead) Foday Sankoh of Sierra Leone,Babangida of Nigeria etc. All these people have hundreds of consignments deposited with Standard securities. Their foreign partners friends and relatives are claiming most of these consignments, but a lot of them are lying here unclaimed for as much as fifteen years. Nobody has ever come for them because in most cases, the documents! of deposits are never available to any body except the depositors but most of them are dead. Some time last year, Standard Security management changed the procedure of claims of consignments. As soon as one is able to supply answers all the test questions as contained in the secret files of any consignment, They consignment will be released to you upon demand from our safe storage. More than one hundred and twenty consignments belonging to Gen.Abacha, and Mobutu have been claimed in the past months. This is why I am soliciting for your co-operation and assistance. Gen. Abacha has 85 consignments deposited with several different names and codes. 35 have been claimed in the past 6 months. Since he died, the first son also died in a plane crash and the second son is facing trial for murderand embezzlement. The family members are under restricted arrest without communication.And can no longer tourch or ask of this consignments as the eyes of their goverment is on them and their consignment with us has overstayed their grace period I have finished every arrangement for you to come and claim consignment NO: 1201 containing US$ 9,000,000.00 and consignment NO: 1200 containing US$ 12,000,000.00. My duty is to supply you with all the information and documents by fax or email which you will use to deal directly with the management. The procedure is simple; You will apply officially to the Director of Operations of Standard Securities for the release of consignments NO; 1200 and NO.1201. They will demand some documents and Secret Codes. Reach me, I will supply you with every detailed information and you will fax it to them. As soon as they confirm it correct, they will invite you for collection. If you do not want to come to Accra, you can arrange with them to transfer the consignment to anywhere but united States of America on agreement. Nobody should ever know that I am involved in this deal except the Lawyer who will write an Agreement for us. I will suggest upon conclusion, 30% goes to you and 70% for me. At the successful completion of the deal, you will arrange for me and family to come over to your country. I am assuring you that this business have been arranged for years now. It is very secure and risk free. Reach me on kofigrey@yahoo.com for further explanation and directives on the procedure, God bless you. Yours Sincerely, GREY KOFI Get back to me via:kofigrey@yahoo.com From akuchlin@mems-exchange.org Thu Apr 18 17:35:00 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Thu, 18 Apr 2002 12:35:00 -0400 Subject: [Python-Dev] Significance of informational PEPs Message-ID: With Barry's approval, I've just checked in a minor change to PEP 1 that clarifies the significance of information PEPs: diff -u -r1.31 pep-0001.txt --- pep-0001.txt 5 Apr 2002 19:42:56 -0000 1.31 +++ pep-0001.txt 18 Apr 2002 16:26:32 -0000 @@ -35,7 +35,9 @@ new feature or implementation for Python. An informational PEP describes a Python design issue, or provides general guidelines or information to the Python community, but does not propose a new - feature. + feature. Informational PEPs do not represent a Python community + consensus or recommendation, so users and implementors are free to + ignore informational PEPs or follow their advice. If people disagree about this view of informational PEPs as purely advisory, please speak up now so we can discuss this issue. (This came up because of the recently-posted PEP 272. Some people on the python-crypto list don't like PEP 272's interface at all, and other people think the whole idea of a standard API for block encryption is useless. It seems impossible to come to consensus, so I wanted to underscore that informational PEPs don't bind people to the interfaces they describe.) --amk (www.amk.ca) I was continually connected with the whole world and never got any rest. At the moment, I spend only a few hours weekly on the net, that's just better for me. -- Peter Greenaway, in an interview in _Page_, May 1999 From aahz@pythoncraft.com Thu Apr 18 18:14:32 2002 From: aahz@pythoncraft.com (Aahz) Date: Thu, 18 Apr 2002 13:14:32 -0400 Subject: [Python-Dev] Significance of informational PEPs In-Reply-To: References: Message-ID: <20020418171432.GA14572@panix.com> On Thu, Apr 18, 2002, Andrew Kuchling wrote: > > + feature. Informational PEPs do not represent a Python community > + consensus or recommendation, so users and implementors are free to > + ignore informational PEPs or follow their advice. > > If people disagree about this view of informational PEPs as purely > advisory, please speak up now so we can discuss this issue. > > (This came up because of the recently-posted PEP 272. Some people on > the python-crypto list don't like PEP 272's interface at all, and > other people think the whole idea of a standard API for block > encryption is useless. It seems impossible to come to consensus, so I > wanted to underscore that informational PEPs don't bind people to the > interfaces they describe.) Seems to me that if an informational PEP gets BDFL approval, it's binding, but possibly not enforced. I'm not sure what the point of an informational PEP that has no consensus is, though; shouldn't such documentation be spread by other means? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ What if there were no rhetorical questions? From jeremy@zope.com Thu Apr 18 18:20:30 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Thu, 18 Apr 2002 13:20:30 -0400 Subject: [Python-Dev] Significance of informational PEPs In-Reply-To: References: Message-ID: <15551.94.896174.164259@slothrop.zope.com> >>>>> "AMK" == Andrew Kuchling writes: AMK> + Informational PEPs do not represent a Python community AMK> + consensus or recommendation, so users and implementors are free AMK> + to ignore informational PEPs or follow their advice. AMK> If people disagree about this view of informational PEPs as AMK> purely advisory, please speak up now so we can discuss this AMK> issue. This seems to match informational RFCs pretty well. It might be interesting to see what the IETF says about informational RFCs. I might word the disclaimer a little less strongly: "Informational PEPs do not necessarily represent a Python community consensus or recommendation." For things like the DB-API, I think there is fairly broad consensus. Rather than claiming there isn't, an informational PEP could just say what its status is. Jeremy From guido@python.org Thu Apr 18 18:40:14 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 18 Apr 2002 13:40:14 -0400 Subject: [Python-Dev] (no subject) In-Reply-To: Your message of "Thu, 18 Apr 2002 17:09:14 BST." <31575A892FF6D1118F5800600846864DCBCFF3@intrepid> References: <31575A892FF6D1118F5800600846864DCBCFF3@intrepid> Message-ID: <200204181740.g3IHeEo06424@odiug.zope.com> > Me too. But what about: > > "oofoo".strip("foo") => "" > "oofoo".strip(["foo", "bar"]) => "oo" > > I.e., strip could accept *any* sequence (rather than just a string) as its > optional parameter, and strip all occurrences of any item in that sequence. > > More flexible, but I can't think of a good use case, to be honest. That's why I'm going to call YAGNI on it. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Apr 18 18:45:48 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 18 Apr 2002 13:45:48 -0400 Subject: [Python-Dev] Significance of informational PEPs In-Reply-To: Your message of "Thu, 18 Apr 2002 12:35:00 EDT." References: Message-ID: <200204181745.g3IHjmb06492@odiug.zope.com> > If people disagree about this view of informational PEPs as purely > advisory, please speak up now so we can discuss this issue. I think *some* informational PEPs (e.g. PEP 1 itself :-) are actually more like standards, while others are, indeed, informational. Maybe we need separate categories? Or simply call the standard ones standard, even if they don't define some Python feature? > (This came up because of the recently-posted PEP 272. Some people on > the python-crypto list don't like PEP 272's interface at all, and > other people think the whole idea of a standard API for block > encryption is useless. It seems impossible to come to consensus, so I > wanted to underscore that informational PEPs don't bind people to the > interfaces they describe.) I certainly prefer that over trying to lobby me for a pronouncement. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas.heller@ion-tof.com Thu Apr 18 19:03:42 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 18 Apr 2002 20:03:42 +0200 Subject: [Python-Dev] A better index for Python manuals? Message-ID: <19f001c1e703$60695b00$475afea9@thomasnotebook> Would it be possible to create a "better" index for Python manuals? For example, show the page-number/link in bold where an api-function is defined instead of only mentioned... There are several tools now which scan the HTML index to build lists of keywords, they can't get too many information from entries like this: Py_Finalize(), [Link], [Link], [Link], [Link], [Link] Thomas From jeremy@zope.com Thu Apr 18 19:04:42 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Thu, 18 Apr 2002 14:04:42 -0400 Subject: [Python-Dev] Significance of informational PEPs In-Reply-To: <200204181745.g3IHjmb06492@odiug.zope.com> References: <200204181745.g3IHjmb06492@odiug.zope.com> Message-ID: <15551.2746.155325.950007@slothrop.zope.com> I just glaned at RFC 2026, which describes in Internet standards process including the various kinds of RFCs. They have two designations that are related to our informational PEPs: informational and experimental. It would be good for us to have experimental PEPs, I think. The three name optimization PEPs fall in this category at the moment. The informational spec seems to be a broad catch-all category, just like for PEPs: An "Informational" specification is published for the general information of the Internet community, and does not represent an Internet community consensus or recommendation. The Informational designation is intended to provide for the timely publication of a very broad range of responsible informational documents from many sources, subject only to editorial considerations and to verification that there has been adequate coordination with the standards process (see section 4.2.3). BTW, I didn't like the title of the structured text PEP because it had standard in its title, which seems contradictory for an informational PEP. Jeremy From guido@python.org Thu Apr 18 19:26:56 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 18 Apr 2002 14:26:56 -0400 Subject: [Python-Dev] Significance of informational PEPs In-Reply-To: Your message of "Thu, 18 Apr 2002 14:04:42 EDT." <15551.2746.155325.950007@slothrop.zope.com> References: <200204181745.g3IHjmb06492@odiug.zope.com> <15551.2746.155325.950007@slothrop.zope.com> Message-ID: <200204181826.g3IIQuC06727@odiug.zope.com> > It would be good for us to have experimental PEPs, I think. The three > name optimization PEPs fall in this category at the moment. +1 > BTW, I didn't like the title of the structured text PEP because it had > standard in its title, which seems contradictory for an informational > PEP. In any case that should also be an experimental one. --Guido van Rossum (home page: http://www.python.org/~guido/) From Jack.Jansen@oratrix.com Thu Apr 18 19:39:16 2002 From: Jack.Jansen@oratrix.com (Jack Jansen) Date: Thu, 18 Apr 2002 20:39:16 +0200 Subject: [Python-Dev] autoheader warning for WITH_UNIVERSAL_NEWLINES In-Reply-To: <200204161241.g3GCfFY20191@pcp742651pcs.reston01.va.comcast.net> Message-ID: <9611C546-52FB-11D6-8E61-003065517236@oratrix.com> On dinsdag, april 16, 2002, at 02:41 , Guido van Rossum wrote: > autoheader is used to create the pyconfig.h.in file. It's run by the > Makefile when you say "make autoconf". A common mistake when adding a > new configure check or option is to add the desired symbol directly > into pyconfig.h.in. The correct solution is currently to use a 3-arg > AC_DEFINE call as Martin showed; until last week, it was to add a few > lines to acconfig.h (which no longer exists). Bah, I am not happy with this. This makes it even more difficult to create the correct pyconfig.h for MacPython. How is updating pyconfig.h handled for Windows (and other non-autoconf platforms)? > Docs for autoconf and autoheader are in info form. If you're still a > vi(m) user :-), "info autoheader" at the shell command line will tell > you more. I'm a BBEdit user, of course, the power of emacs without the finger cramps:-) -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From akuchlin@mems-exchange.org Thu Apr 18 19:50:32 2002 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Thu, 18 Apr 2002 14:50:32 -0400 Subject: [Python-Dev] Significance of informational PEPs In-Reply-To: <200204181745.g3IHjmb06492@odiug.zope.com> References: <200204181745.g3IHjmb06492@odiug.zope.com> Message-ID: <20020418185032.GE1262@ute.mems-exchange.org> On Thu, Apr 18, 2002 at 01:45:48PM -0400, Guido van Rossum wrote: >I think *some* informational PEPs (e.g. PEP 1 itself :-) are actually >more like standards, while others are, indeed, informational. Maybe >we need separate categories? Or simply call the standard ones >standard, even if they don't define some Python feature? PEP 1 doesn't specify anything related to programming or documenting Python, though. I don't think it's worth defining a new category to distinguish PEP 1 from PEP 248 (Database API) or PEP 272. Regarding Jeremy's suggestion of introducing an "Experimental" category: what's the distinction be between "Experimental" and "Informational", if neither status carries any implication that users should conform to the PEP? For example, would PEP 272 be Experimental or Informational, and why? Aahz wrote: >I'm not sure what the point of an informational >PEP that has no consensus is, though; shouldn't such documentation be >spread by other means? PEPs seem nicely suited for this, and it's how RFCs are used, too. I could just stick it on a random Web page someplace, but PEPs, like RFCs, are much less likely to vanish and are much shorter to reference. --amk (www.amk.ca) Given the choice between a good text editor and a good source control system, i'll take the source control, and use "cat" to write my code. -- Greg Wilson, at IPC9 From tim.one@comcast.net Thu Apr 18 20:05:32 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 18 Apr 2002 15:05:32 -0400 Subject: [Python-Dev] autoheader warning for WITH_UNIVERSAL_NEWLINES In-Reply-To: <9611C546-52FB-11D6-8E61-003065517236@oratrix.com> Message-ID: [Jack Jansen] > ... > How is updating pyconfig.h handled for Windows Via by-hand editing of PC\pyconfig.h. > (and other non-autoconf platforms)? Dunno. Any two distinct flavors of Win32 to date are typically much more compatible with each other than are two distinct flavors of Unix; an autoconf-like gimmick would be (way) more trouble than it's worth on Windows. From guido@python.org Thu Apr 18 20:06:54 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 18 Apr 2002 15:06:54 -0400 Subject: [Python-Dev] autoheader warning for WITH_UNIVERSAL_NEWLINES In-Reply-To: Your message of "Thu, 18 Apr 2002 20:39:16 +0200." <9611C546-52FB-11D6-8E61-003065517236@oratrix.com> References: <9611C546-52FB-11D6-8E61-003065517236@oratrix.com> Message-ID: <200204181906.g3IJ6sV06984@odiug.zope.com> > Bah, I am not happy with this. This makes it even more difficult > to create the correct pyconfig.h for MacPython. Really? You have all the templates at your fingertips in pyconfig.h.in. > How is updating pyconfig.h handled for Windows (and other > non-autoconf platforms)? Painfully, and erratically. Tim complains that a certain unit test fails or is skipped, I determine that it's because a certain feature is not configured, and then he adds it. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Apr 18 20:10:07 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 18 Apr 2002 15:10:07 -0400 Subject: [Python-Dev] Significance of informational PEPs In-Reply-To: Your message of "Thu, 18 Apr 2002 14:50:32 EDT." <20020418185032.GE1262@ute.mems-exchange.org> References: <200204181745.g3IHjmb06492@odiug.zope.com> <20020418185032.GE1262@ute.mems-exchange.org> Message-ID: <200204181910.g3IJA7607012@odiug.zope.com> > PEP 1 doesn't specify anything related to programming or documenting > Python, though. I don't think it's worth defining a new category to > distinguish PEP 1 from PEP 248 (Database API) or PEP 272. The "MetaPEP" category? > Regarding Jeremy's suggestion of introducing an "Experimental" > category: what's the distinction be between "Experimental" and > "Informational", if neither status carries any implication that users > should conform to the PEP? For example, would PEP 272 be Experimental > or Informational, and why? Most PEPs start their life as experimental. Then they either become Standard (with Draft status in between), or Rejected/Withdrawn, or Informational if no consensus is reached but the PEP is still useful (I expect the docstring PEPs to go here). Some PEPs start out Informational, if they are descriptive of historical truths. > Aahz wrote: > >I'm not sure what the point of an informational > >PEP that has no consensus is, though; shouldn't such documentation be > >spread by other means? > > PEPs seem nicely suited for this, and it's how RFCs are used, too. I > could just stick it on a random Web page someplace, but PEPs, like > RFCs, are much less likely to vanish and are much shorter to > reference. Agreed. For most practical purposes, PEPs are isomorphic with RFCs, and I expect that just as the RFC process became more formalized as the IETF community grew, we'll have to become more anal retentive about PEP rules too. --Guido van Rossum (home page: http://www.python.org/~guido/) From andy@reportlab.com Thu Apr 18 21:00:28 2002 From: andy@reportlab.com (Andy Robinson) Date: Thu, 18 Apr 2002 21:00:28 +0100 Subject: [Python-Dev] Transaction In-Reply-To: <20020418191002.20481.21763.Mailman@mail.python.org> Message-ID: > claim consignment NO: 1201 containing US$ 9,000,000.00 > and consignment NO: 1200 containing US$ 12,000,000.00. > I will suggest upon conclusion, 30% goes to you and > 70% for me. At the successful completion of the deal, > you will arrange for me and family to come over to > your country. If you can persuade the INS that this guy is a uniquely qualified release manager and get hima green card, it could fund a LOT of backporting and release management :-) - Andy From tim.one@comcast.net Thu Apr 18 21:14:50 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 18 Apr 2002 16:14:50 -0400 Subject: [Python-Dev] autoheader warning for WITH_UNIVERSAL_NEWLINES In-Reply-To: <200204181906.g3IJ6sV06984@odiug.zope.com> Message-ID: [Guido] > Painfully, and erratically. Tim complains that a certain unit test > fails or is skipped, I determine that it's because a certain feature > is not configured, and then he adds it. Na, I don't recall that ever happening, and neither does PC\pyconfig.h's CVS history . It's true that universal newlines isn't yet enabled in PC\pyconfig.h. From just@letterror.com Thu Apr 18 21:20:07 2002 From: just@letterror.com (Just van Rossum) Date: Thu, 18 Apr 2002 22:20:07 +0200 Subject: [Python-Dev] autoheader warning for WITH_UNIVERSAL_NEWLINES In-Reply-To: Message-ID: [Guido] > Painfully, and erratically. Tim complains that a certain unit test > fails or is skipped, I determine that it's because a certain feature > is not configured, and then he adds it. [Tim] > Na, I don't recall that ever happening, and neither does PC\pyconfig.h's CVS > history . It's true that universal newlines isn't yet enabled in > PC\pyconfig.h. I think that Guido remembers *Jack's* complaints... Just From niemeyer@conectiva.com Thu Apr 18 21:28:27 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Thu, 18 Apr 2002 17:28:27 -0300 Subject: [Python-Dev] Re: tarfile In-Reply-To: <10835036820.20020418212353@gustaebel.de> References: <20020413001935.A1501@ibook.distro.conectiva> <1128502796.20020413174605@gustaebel.de> <20020413155750.A9454@ibook.distro.conectiva> <1299829994.20020414193125@gustaebel.de> <20020416184443.A28648@ibook.distro.conectiva> <10835036820.20020418212353@gustaebel.de> Message-ID: <20020418172826.A6835@ibook.distro.conectiva> Hi Lars! > > Itamar told me about some changes he submitted to zipfile, and weren't > > accepted by lack of documentation and unittests. Here is an excerpt of > > his message: > [...] > > Could you please review it!? > > I had a look at it and IMO it's the right way. Too bad that the patch > did not succeed. I started to include a similar method today replacing > the readstr() method. The same will follow with writestr(). Thanks! > But OTOH this does not solve the problem I described in my last message: > >> [...] > >> When using tarfile the user assumes that all of > >> the meta-informations of a file like mode, ownership, modification > >> time etc. are extracted as well, for example when restoring a backup. > > We should find a solution to that, most of the code of tarfile deals > with these particular tasks... What do you think about keeping the current zipfile semantics of read/write returning strings, and using your suggesion of extract() and add() to implement the new functionality of dumping to disk? We could even have some kind of Dumper class, which would be responsible for that task, and could be replaced by the user if necessary (personaly, I'm not against something like "read = extractstr" as well). > > I understand your position, and agree with you. Nevertheless, we should > > let the interface as close as *possible*. If zipfile is not well > > designed, I'd like to change it as well. > >From my (insignificant) point of view, I would not care too much about > standardization in this particular case, because the "standard" > introduced by the zipfile module is *very* *very* limited. Yes, zipfile.py interface's is not a nice example for a global interface for file storing. > I think we don't do ourselves a favour if we take zipfile's interface > as a basis. And I think we don't do the user a favour because he would > always tend towards mixing up both interfaces, which would both be > somehow similar but OTOH fundamentally different. > > Wouldn't it be good to do it the opposite way round and create a solid > interface for tarfile, which could be used as a basis for a > new/patched zipfile or perhaps some day for other archiver-related > modules? Yes, it'd be nice indeed, but I wouldn't like to see, for example, a cpio handler with a different interface in the future, because the current ones don't fit well for the task. You told me you were willing to write a PEP. Is that PEP about tarfile specific issues, or a general interface for that kind of job? > I mention again the small TarFileCompat class I wrote within 5 > minutes, which perfectly emulates a ZipFile class. Try that > the other way 'round and you're sold down the river... I understand.. > I apologize deeply for my obstinacy.... But I can't help myself. > Please tell me, if I exaggerate or if I am wrong. But at the moment I > don't have any idea how to be "as close as possible" to zipfile, > sorry. Again, you've done a great job with tarfile so far, and I'm thankful for that. I'm sure we can handle those issues and still have something you're proud of. Btw, I've notice TarInfo is using class attributes on initialization. Wouldn't instance attributes reflect better their meaning? Thanks! -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From DavidA@ActiveState.com Thu Apr 18 22:15:59 2002 From: DavidA@ActiveState.com (David Ascher) Date: Thu, 18 Apr 2002 14:15:59 -0700 Subject: [Python-Dev] Re: tarfile References: <20020413001935.A1501@ibook.distro.conectiva> <1128502796.20020413174605@gustaebel.de> <20020413155750.A9454@ibook.distro.conectiva> <1299829994.20020414193125@gustaebel.de> <20020416184443.A28648@ibook.distro.conectiva> <10835036820.20020418212353@gustaebel.de> <20020418172826.A6835@ibook.distro.conectiva> Message-ID: <3CBF378F.7090600@ActiveState.com> Gustavo Niemeyer wrote: >What do you think about keeping the current zipfile semantics of >read/write returning strings, and using your suggesion of extract() and >add() to implement the new functionality of dumping to disk? We could >even have some kind of Dumper class, which would be responsible for >that task, and could be replaced by the user if necessary (personaly, >I'm not against something like "read = extractstr" as well). > I just want to point out that, when dealing with really large files (which are prime candidates for zipping =), there are significant benefits to being able to look at parts of a file in a zip archive without having to dump the file to disk. If you look at the gzip module, for example, you'll see that it provides a file-like interface to the data, without requiring it to _be_ a file. This may be obvious to you, but the above paragraph made me want to be explicit. --david ascher From mhammond@skippinet.com.au Fri Apr 19 01:58:04 2002 From: mhammond@skippinet.com.au (Mark Hammond) Date: Fri, 19 Apr 2002 10:58:04 +1000 Subject: [Python-Dev] A better index for Python manuals? In-Reply-To: <19f001c1e703$60695b00$475afea9@thomasnotebook> Message-ID: > Would it be possible to create a "better" index for Python manuals? > > For example, show the page-number/link in bold where an api-function > is defined instead of only mentioned... > > There are several tools now which scan the HTML index to build > lists of keywords, they can't get too many information from > entries like this: > > Py_Finalize(), [Link], [Link], [Link], [Link], [Link] I also have a problem with a few entries - eg, PYTHONPATH - it gets lots of index entries, but many point to consecutive paragraphs in the same document. But I fear this is off-topic - Fred, what is the process for building the HTML index, and should we discuss this on the doc-sig? Mark. From skip@pobox.com Fri Apr 19 02:43:45 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 18 Apr 2002 20:43:45 -0500 Subject: [Python-Dev] CVS/syncmail/SF Message-ID: <15551.30289.788967.628077@12-248-41-177.client.attbi.com> Sorry for the not-quite-Python-related question, but where do I checkout syncmail on SF if I want to use it like so from CVSROOT/loginfo: DEFAULT $CVSROOT/CVSROOT/syncmail %{sVv} largefiles-checkins@lists.sourceforge.net In the "getting email from CVS" page it says: After you've handled the checkout a working copy of syncmail into the repository, you need to modify your CVSROOT/loginfo file to tell CVS to invoke syncmail for all the interesting events CVS is willing to tell us about. I poked around all the SF docs that looked obvious without any luck. I also ssh'd into shell.sf.net and largefiles.sf.net but found no /cvsroot/largefiles directory on either. I posted a SF support request a few hours ago, but haven't heard anything from them. From the looks of the number of support requests open, I anticipate a lengthy wait. Where am I supposed to checkout a working copy of syncmail? I presume it has to be on a SF machine somewhere, but I've been unable to figure out where. Thx, Skip From neal@metaslash.com Fri Apr 19 04:07:45 2002 From: neal@metaslash.com (Neal Norwitz) Date: Thu, 18 Apr 2002 23:07:45 -0400 Subject: [Python-Dev] CVS/syncmail/SF References: <15551.30289.788967.628077@12-248-41-177.client.attbi.com> Message-ID: <3CBF8A01.D8CF707B@metaslash.com> Skip Montanaro wrote: > > Sorry for the not-quite-Python-related question, but where do I checkout > syncmail on SF if I want to use it like so from CVSROOT/loginfo: > > DEFAULT $CVSROOT/CVSROOT/syncmail %{sVv} largefiles-checkins@lists.sourceforge.net > > In the "getting email from CVS" page it says: ... It's right above that, here's the link on the page: http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/mailman/CVSROOT/syncmail Or the real file to get: http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/*checkout*/mailman/CVSROOT/syncmail?rev=HEAD&content-type=text/plain Neal From fdrake@acm.org Fri Apr 19 04:19:24 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 18 Apr 2002 23:19:24 -0400 Subject: [Python-Dev] CVS/syncmail/SF In-Reply-To: <15551.30289.788967.628077@12-248-41-177.client.attbi.com> References: <15551.30289.788967.628077@12-248-41-177.client.attbi.com> Message-ID: <15551.36028.883840.835598@grendel.zope.com> Skip Montanaro writes: > Where am I supposed to checkout a working copy of syncmail? I presume it > has to be on a SF machine somewhere, but I've been unable to figure out > where. syncmail is currently maintained in a separate SF project: http://sourceforge.net/projects/cvs-syncmail/ Get a checkout from that project, copy the syncmail script into the checkout CVSROOT in the project you want to add it to, and check it in there. Once all the commits are made to the files in CVSROOT for the target project, it will be ready. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From barry@zope.com Fri Apr 19 04:44:28 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 18 Apr 2002 23:44:28 -0400 Subject: [Python-Dev] CVS/syncmail/SF References: <15551.30289.788967.628077@12-248-41-177.client.attbi.com> Message-ID: <15551.37532.161628.927425@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: SM> Where am I supposed to checkout a working copy of syncmail? I SM> presume it has to be on a SF machine somewhere, but I've been SM> unable to figure out where. syncmail's got its own SF project page now: http://sourceforge.net/projects/cvs-syncmail Note that we had to call the project "cvs-syncmail" since "syncmail" was taken for something unrelated. -Barry From martin@v.loewis.de Fri Apr 19 08:04:59 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 19 Apr 2002 09:04:59 +0200 Subject: [Python-Dev] autoheader warning for WITH_UNIVERSAL_NEWLINES In-Reply-To: <9611C546-52FB-11D6-8E61-003065517236@oratrix.com> References: <9611C546-52FB-11D6-8E61-003065517236@oratrix.com> Message-ID: Jack Jansen writes: > Bah, I am not happy with this. This makes it even more difficult to > create the correct pyconfig.h for MacPython. How is updating > pyconfig.h handled for Windows (and other non-autoconf platforms)? Can you elaborate what "this" is? pyconfig.h.in always was a generated file; it always started with the line /* pyconfig.h.in. Generated from configure.in by autoheader. */ So when you manually edited pyconfig.h.in, you *could* have known better. Regards, Martin From mwh@python.net Fri Apr 19 13:38:45 2002 From: mwh@python.net (Michael Hudson) Date: 19 Apr 2002 13:38:45 +0100 Subject: [Python-Dev] Adding Optik to 2.3 standard library References: <200204170417.g3H4H8o20585@localhost.localdomain> <200204171209.g3HC9YA29287@pcp742651pcs.reston01.va.comcast.net> <20020417082126.A28098@mems-exchange.org> <20020417225621.GA12027@panix.com> <08391204600885@aluminium.rcp.co.uk> Message-ID: <2mofgfzwve.fsf@starship.python.net> Duncan Booth writes: > On 17 Apr 2002, Aahz wrote: > > > Side note for those unfamiliar with this bit of Windows lore: all > > versions of Windows claiming POSIX compatibility ship the pax utility, > > and thus can read and write tar files. > > Not only that, but if you copy/rename the Windows pax.exe program to > tar.exe or cpio.exe it changes its command line options to match. So all > recent versions of Windows ship with tar and cpio included (sort of). Oh, so thats why some of the files inside installer bundles on MacOS have the extension .pax.gz. Learn something every day. Cheers, M. -- languages shape the way we think, or don't. -- Erik Naggum, comp.lang.lisp From Jack.Jansen@cwi.nl Fri Apr 19 15:05:30 2002 From: Jack.Jansen@cwi.nl (Jack Jansen) Date: Fri, 19 Apr 2002 16:05:30 +0200 Subject: [Python-Dev] autoheader warning for WITH_UNIVERSAL_NEWLINES In-Reply-To: <200204181906.g3IJ6sV06984@odiug.zope.com> Message-ID: <823914E4-539E-11D6-A543-0030655234CE@cwi.nl> On Thursday, April 18, 2002, at 09:06 , Guido van Rossum wrote: >> Bah, I am not happy with this. This makes it even more difficult >> to create the correct pyconfig.h for MacPython. > > Really? You have all the templates at your fingertips in > pyconfig.h.in. You're right, so all isn't lost. Now the only prooblem is that the order of the defines in pyconfig.h.in has changed again:-( -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From mejiaspolonio@teleline.es Fri Apr 19 19:22:21 2002 From: mejiaspolonio@teleline.es (CERAMICA MEJIAS POLONIO C.B. (SPAIN)) Date: Fri, 19 Apr 2002 20:22:21 +0200 Subject: [Python-Dev] POTTERY Message-ID: <006a01c1e7cf$26a0bf40$15ef63d5@mio> This is a multi-part message in MIME format. ------=_NextPart_000_0067_01C1E7DF.E92F50C0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Dear Sirs Our company=B4s name is "Cer=E1mica Mej=EDas Polonio". We manufacture = hand-made pottery for decoration. We would like to offer you our articles which you can see on our = web-site www.myempresa.com/mejiaspolonio=20 We are already exporting to some European and Asian countries where our = articles are very successful due to their good finishing and difference = in style. We hope you like our articles. If you want to contact us you can do it = at: "Cer=E1mica Mej=EDas Polonio C.B." C/Carrera Baja,22 14540 La Rambla (Cordoba) Spain. Phone/fax: +34 957 682 280 Translator=B4s contact: +34 650 06 02 79 e-mail: mejiaspolonio@teleline.es web: www.myempresa.com/mejiaspolonio=20 Waiting for your news.=20 Yours Faithfully. ------=_NextPart_000_0067_01C1E7DF.E92F50C0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

 

Dear Sirs

Our company=B4s name is "Cer=E1mica Mej=EDas Polonio". We manufacture = hand-made=20 pottery for decoration.

We would like to offer you our articles which you can see on our = web-site=20 www.myempresa.com/mejiaspolonio

We are already exporting to some European and Asian countries where = our=20 articles are very successful due to their good finishing and difference = in=20 style.

We hope you like our articles. If you want to contact us you can do = it=20 at:

"Cer=E1mica Mej=EDas Polonio C.B."

C/Carrera Baja,22

14540 La Rambla (Cordoba)

Spain.

Phone/fax: +34 957 682 280

Translator=B4s contact: +34 650 06 02 79

e-mail: mejiaspolonio@teleline.es

web: www.myempresa.com/mejiaspolonio

Waiting for your news.

Yours Faithfully.

------=_NextPart_000_0067_01C1E7DF.E92F50C0-- From martin@v.loewis.de Fri Apr 19 21:29:42 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 19 Apr 2002 22:29:42 +0200 Subject: [Python-Dev] Adding Optik to 2.3 standard library In-Reply-To: <2mofgfzwve.fsf@starship.python.net> References: <200204170417.g3H4H8o20585@localhost.localdomain> <200204171209.g3HC9YA29287@pcp742651pcs.reston01.va.comcast.net> <20020417082126.A28098@mems-exchange.org> <20020417225621.GA12027@panix.com> <08391204600885@aluminium.rcp.co.uk> <2mofgfzwve.fsf@starship.python.net> Message-ID: Michael Hudson writes: > > Not only that, but if you copy/rename the Windows pax.exe program to > > tar.exe or cpio.exe it changes its command line options to match. So all > > recent versions of Windows ship with tar and cpio included (sort of). > > Oh, so thats why some of the files inside installer bundles on MacOS > have the extension .pax.gz. Learn something every day. This is getting off-topic, but I'd like to point out that introduction of the "pax" utility was the solution to the "tar wars" :-) Regards, Martin From martin@v.loewis.de Fri Apr 19 21:27:51 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 19 Apr 2002 22:27:51 +0200 Subject: [Python-Dev] autoheader warning for WITH_UNIVERSAL_NEWLINES In-Reply-To: <823914E4-539E-11D6-A543-0030655234CE@cwi.nl> References: <823914E4-539E-11D6-A543-0030655234CE@cwi.nl> Message-ID: Jack Jansen writes: > You're right, so all isn't lost. Now the only prooblem is that the > order of the defines in pyconfig.h.in has changed again:-( Why is that a problem (even if just the only one)? Regards, Martin From martin@v.loewis.de Fri Apr 19 21:52:26 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 19 Apr 2002 22:52:26 +0200 Subject: [Python-Dev] PEP 263 update (source encodings) Message-ID: Since Mr. Suzuki proposed a patch for phase 2 of PEP 263, I've been thinking that the implementation could go right away to phase two; skipping my implementation of phase 1. Still, providing the transitional warning of phase 1 is desirable, and indeed possible with this implementation. So after discussion with MAL, I have updated the PEP to reflect this strategy. There are only minimal user-visible changes: - more encodings can be supported even in phase 1, in particular those based on ISO 2022 (provided that the proper codecs are available) - it is now an error if the declared encoding and the text do not match. If there is no encoding declared, it is still only a warning if you then use non-ASCII bytes. More interesting is the change to the implementation strategy: the parser will create a codecs.StreamReader to obtain the input, and will convert each line to UTF-8; parsing then operates on the UTF-8 strings. This has the advantage that all lexical processing can still use the same char*, interpreted as ASCII, as before. The declared encoding is preserved (as in my stage 1 implementation), in order to recode string literals back to the original encoding. Anybody familiar with the parser is encouraged to review this code; see http://python.org/sf/534304 http://python.org/peps/pep-0263.html Regards, Martin From tim.one@comcast.net Sun Apr 21 04:12:25 2002 From: tim.one@comcast.net (Tim Peters) Date: Sat, 20 Apr 2002 23:12:25 -0400 Subject: [Python-Dev] test_inspect failure Message-ID: I'm seeing test_inspect.py fail on Windows, but only when run under regrtest: C:\Code\python\PCbuild>python ../lib/test/regrtest.py -v test_inspect.py test_inspect test test_inspect crashed -- exceptions.IndexError: list index out of range Traceback (most recent call last): File "../lib/test/regrtest.py", line 315, in runtest File "../lib/test\test_inspect.py", line 167, in ? git.abuse(7, 8, 9) File "@test", line 39, in abuse self.argue(a, b, c) File "@test", line 46, in argue self.tr = inspect.trace() File "C:\CODE\PYTHON\lib\inspect.py", line 782, in trace File "C:\CODE\PYTHON\lib\inspect.py", line 763, in getinnerframes File "C:\CODE\PYTHON\lib\inspect.py", line 717, in getframeinfo File "C:\CODE\PYTHON\lib\inspect.py", line 413, in findsource IndexError: list index out of range 1 test failed: test_inspect C:\Code\python\PCbuild> It's certainly curious that the traceback is missing all the code lines from inspect.py! Even curiouser, when run directly, no problem: C:\Code\python\PCbuild>python ../lib/test/test_inspect.py C:\Code\python\PCbuild> I'm using stock CVS, + defining WITH_UNIVERSAL_NEWLINES. This is suspicious, because when run under regrtest, test_inspect gets imported, and then universal newlines kicks in (if I understood the PEP); I'm not clear on whether universal newlines kicks in when a file is run directly. Is anyone else with universal newlines (or without, for that matter ...) seeing test_inspect fail? From tim.one@comcast.net Sun Apr 21 05:22:39 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 21 Apr 2002 00:22:39 -0400 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Objects unicodeobject.c,2.139,2.140 In-Reply-To: <3CC1888D.D434DDB3@lemburg.com> Message-ID: I expect Martin checked in this change because of the unhappy hours he spent determining that the previous two versions of this function wrote beyond the memory they allocated. Since the most recent version still didn't bother to assert that it wasn't writing out of bounds, I can't blame Martin for checking in a version that does so assert; since I spent hours on this too, and this function has a repeated history of bad memory behavior, I viewed the version Martin replaced as unacceptable. However, the slowdown on large strings isn't attractive, and the previous version could easily enough have asserted its memory correctness. > -----Original Message----- > From: python-checkins-admin@python.org > [mailto:python-checkins-admin@python.org]On Behalf Of M.-A. Lemburg > Sent: Saturday, April 20, 2002 11:26 AM > To: loewis@sourceforge.net > Cc: python-checkins@python.org > Subject: Re: [Python-checkins] python/dist/src/Objects > unicodeobject.c,2.139,2.140 > > > loewis@sourceforge.net wrote: >> >> Update of /cvsroot/python/python/dist/src/Objects >> In directory usw-pr-cvs1:/tmp/cvs-serv30961 >> >> Modified Files: >> unicodeobject.c >> Log Message: >> Patch #495401: Count number of required bytes for encoding UTF-8 >> before allocating the target buffer. > > Martin, please back out this change again. We have discussed this > quite a few times and I am against using your strategy since > it introduces a performance hit which does not relate to the > gained advantage of (temporarily) using less memory. > > Your timings also show this, so I wonder why you checked in this > patch, e.g. from the patch log: > """ > For the current > CVS (unicodeobject.c 2.136: MAL's change to use a variable > overalloc), I get > > 10 spaces 20.060 > 100 spaces 2.600 > 200 spaces 2.030 > 1000 spaces 0.930 > 10000 spaces 0.690 > 10 spaces, 3 bytes 23.520 > 100 spaces, 3 bytes 3.730 > 200 spaces, 3 bytes 2.470 > 1000 spaces, 3 bytes 0.980 > 10000 spaces, 3 bytes 0.690 > 30 bytes 24.800 > 300 bytes 5.220 > 600 bytes 3.830 > 3000 bytes 2.480 > 30000 bytes 2.230 > > With unicode3.diff (that's the one you checked in), I get > > 10 spaces 19.940 > 100 spaces 3.260 > 200 spaces 2.340 > 1000 spaces 1.650 > 10000 spaces 1.450 > 10 spaces, 3 bytes 21.420 > 100 spaces, 3 bytes 3.410 > 200 spaces, 3 bytes 2.420 > 1000 spaces, 3 bytes 1.660 > 10000 spaces, 3 bytes 1.450 > 30 bytes 22.260 > 300 bytes 5.830 > 600 bytes 4.700 > 3000 bytes 3.740 > 30000 bytes 3.540 > """ > > The only case where your patch is faster is for very short > strings and then only by a few percent, whereas for all > longer strings you get worse timings, e.g. 3.74 seconds > compared to 2.48 seconds -- that's a 50% increase in > run-time ! > > Thanks, > -- > Marc-Andre Lemburg > CEO eGenix.com Software GmbH > ______________________________________________________________________ > Company & Consulting: http://www.egenix.com/ > Python Software: http://www.egenix.com/files/python/ > > > _______________________________________________ > Python-checkins mailing list > Python-checkins@python.org > http://mail.python.org/mailman/listinfo/python-checkins From nas@python.ca Sun Apr 21 05:54:07 2002 From: nas@python.ca (Neil Schemenauer) Date: Sat, 20 Apr 2002 21:54:07 -0700 Subject: [Python-Dev] test_inspect failure In-Reply-To: ; from tim.one@comcast.net on Sat, Apr 20, 2002 at 11:12:25PM -0400 References: Message-ID: <20020420215407.A12829@glacier.arctrix.com> Tim Peters wrote: > Is anyone else with universal newlines (or without, for that matter ...) > seeing test_inspect fail? Works for me on Linux. Neil From tim.one@comcast.net Sun Apr 21 06:24:54 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 21 Apr 2002 01:24:54 -0400 Subject: [Python-Dev] test_inspect failure In-Reply-To: <20020420215407.A12829@glacier.arctrix.com> Message-ID: [Tim] > Is anyone else with universal newlines (or without, for that > matter ...) seeing test_inspect fail? [Neil Schemenauer] > Works for me on Linux. Do you know whether universal newlines is enabled there (e.g., does test_univnewlines skip itself, or run)? I've determined that WITH_UNIVERSAL_NEWLINES is the sole difference between passing and failing on Windows. Alas, I can't yet imagine why, and test_inspect isn't easy to debug. From tim.one@comcast.net Sun Apr 21 06:47:04 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 21 Apr 2002 01:47:04 -0400 Subject: [Python-Dev] test_inspect failure In-Reply-To: Message-ID: OK, it's down to this: >>> f = open('../lib/test/regrtest.py', 'rU') >>> len(f.readlines()) 232 >>> g = open('../lib/test/regrtest.py') >>> len(g.readlines()) 715 >>> IOW, opening in U mode screws up on Windows when universal newlines is enabled. This screws test_inspect.py for reasons so complicated I won't even try to trace the trail here; the bottom line is that lincache.py's updatecache() uses U mode near the end: fp = open(fullname, 'rU') lines = fp.readlines() fp.close() For whatever reason, this only delivers the first 232 lines of regrtest.py. I doubt it's a coincidence this is darned near exactly the first 8192 bytes of the file. From tim.one@comcast.net Sun Apr 21 07:33:59 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 21 Apr 2002 02:33:59 -0400 Subject: [Python-Dev] test_inspect failure In-Reply-To: Message-ID: [Tim] > OK, it's down to this: > > >>> f = open('../lib/test/regrtest.py', 'rU') > >>> len(f.readlines()) > 232 > >>> g = open('../lib/test/regrtest.py') > >>> len(g.readlines()) > 715 > >>> > > IOW, opening in U mode screws up on Windows when universal newlines is > enabled. > ... > For whatever reason, this only delivers the first 232 lines of > regrtest.py. I doubt it's a coincidence this is darned near exactly > the first 8192 bytes of the file. And it's not: 8192 is the initial buffer size used by file_readlines(). The return value of Py_UniversalNewlineFread() isn't documented, so it's hard to know what was intended. It returns the number of bytes in the buffer. But file_readlines() believes it returns the number of bytes actually read from the file. On Windows the former number is less than the latter number, by the count of \r\n pairs shrunk to \n. file_readlines() thus gets back a number smaller than the 8192 it passed in, and so believes the file is exhausted by the first read attempt. I expect the best thing to do is to make ntodo reflect the number of bytes remaining to be filled in the buffer, rather than the number of bytes actually read from the file. From tim.one@comcast.net Sun Apr 21 08:47:55 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 21 Apr 2002 03:47:55 -0400 Subject: [Python-Dev] test_inspect failure In-Reply-To: Message-ID: You can all go to sleep now : I checked in fixes, and universal newlines is enabled now on Windows. Note that the NEWS file needs info about this feature, and about config (how to enable, how to disable). From martin@v.loewis.de Sun Apr 21 10:45:34 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 21 Apr 2002 11:45:34 +0200 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Objects unicodeobject.c,2.139,2.140 In-Reply-To: References: Message-ID: Tim Peters writes: > I expect Martin checked in this change because of the unhappy hours he spent > determining that the previous two versions of this function wrote beyond the > memory they allocated. Since the most recent version still didn't bother to > assert that it wasn't writing out of bounds, I can't blame Martin for > checking in a version that does so assert; since I spent hours on this too, > and this function has a repeated history of bad memory behavior, I viewed > the version Martin replaced as unacceptable. Exactly. I think the most recent version is worse than the one we had before. > However, the slowdown on large strings isn't attractive, and the previous > version could easily enough have asserted its memory correctness. I found the overallocations strategy that this code had just so embarrassing: a single three-byte character will cause an overallocation of three times the memory, which means that the final realloc will certainly truncate lots of bytes. As a result, we are at the mercy of the realloc implementation here: If realloc copies memory (such as Pymalloc might some day) when shrinking buffers, performance will get worse. Since this appears to be religious, I'm backing the patch out. Regards, Martin From skip@mojam.com Sun Apr 21 13:00:19 2002 From: skip@mojam.com (Skip Montanaro) Date: Sun, 21 Apr 2002 07:00:19 -0500 Subject: [Python-Dev] Weekly Python Bug/Patch Summary Message-ID: <200204211200.g3LC0JZ25564@12-248-41-177.client.attbi.com> Bug/Patch Summary ----------------- 233 open / 2432 total bugs (-6) 97 open / 1438 total patches (+4) New Bugs -------- gcc warning in unicodeobject.c (2002-04-15) http://python.org/sf/544248 gcc warning in Modules/readline.c (2002-04-15) http://python.org/sf/544265 test_commands test fails under Cygwin (2002-04-16) http://python.org/sf/544740 Internet Config -666 error (2002-04-17) http://python.org/sf/545059 walk() walks aliases (2002-04-17) http://python.org/sf/545388 corefile: python 2.1.3, zope 2.5.0 (2002-04-17) http://python.org/sf/545410 py-execute-string doesn't work (2002-04-17) http://python.org/sf/545436 IDLE doesn't save 8bit files (2002-04-18) http://python.org/sf/545600 cannot save buffer (2002-04-18) http://python.org/sf/545655 Wrong exception from re.compile() (2002-04-18) http://python.org/sf/545855 regex segfault on Mac OS X (2002-04-19) http://python.org/sf/546059 IDLE calltips cause application error (2002-04-19) http://python.org/sf/546078 readlines() bombs - last line incomplete (2002-04-19) http://python.org/sf/546117 load_false/true() not used (2002-04-19) http://python.org/sf/546156 extend namespace docs to cover LOAD_NAME (2002-04-19) http://python.org/sf/546196 buffer slice type inconsistant (2002-04-20) http://python.org/sf/546434 Windows getpass bug (2002-04-20) http://python.org/sf/546558 Nested anchors in Ref Man index.html (2002-04-20) http://python.org/sf/546579 New Patches ----------- test for patch #543865 & others (2002-04-14) http://python.org/sf/543867 merging sorted sequences (2002-04-15) http://python.org/sf/544113 addition of cmath.arg function (2002-04-16) http://python.org/sf/544909 Janitoring in ConfigParser (2002-04-17) http://python.org/sf/545096 {a,b} in fnmatch.translate (2002-04-17) http://python.org/sf/545150 sgmllib support for additional tag forms (2002-04-17) http://python.org/sf/545300 interactive help in python-mode (2002-04-17) http://python.org/sf/545439 Examples for urllib2 (2002-04-17) http://python.org/sf/545480 make test_linuxaudiodev ignore EBUSY (2002-04-17) http://python.org/sf/545486 patch for 514433 bsddb.dbopen (NULL) (2002-04-18) http://python.org/sf/545523 implementation of Text.dump method (2002-04-19) http://python.org/sf/546244 Closed Bugs ----------- Extension manual: Windows DLL/C++ info needs review (2000-12-08) http://python.org/sf/225003 ConfigParser.options return default opts (2001-07-14) http://python.org/sf/441279 Py closes COM instances not loaded by Py (2001-09-06) http://python.org/sf/459160 test_time fails on Sparc64 (2001-09-10) http://python.org/sf/460357 Build troubles: --with-pymalloc (2001-12-20) http://python.org/sf/495401 smtplib.py AUTH mechanism (2002-01-02) http://python.org/sf/498572 free vars sometimes show up in locals() (2002-01-18) http://python.org/sf/505315 Py_CompileString and tracebacks (2002-01-23) http://python.org/sf/507477 Ellipsis semantics undefined (2002-02-17) http://python.org/sf/518985 compiler/transformer.py STARSTAR doesn't exist (2002-02-24) http://python.org/sf/522274 NameError assigning to class in a func (2002-03-20) http://python.org/sf/532860 SMTPLib sendmail fails on hyphenated adr (2002-03-31) http://python.org/sf/537484 PyType_GenericNew broken (2002-04-08) http://python.org/sf/540965 link to index.html from modindex.html (2002-04-08) http://python.org/sf/541257 IDLE can clobber files (2002-04-09) http://python.org/sf/541730 delattr() causes python to segfault (2002-04-09) http://python.org/sf/541883 urllib.urlopen throws OSError (2002-04-10) http://python.org/sf/541980 allow use of descriptors for __doc_ attr (2002-04-12) http://python.org/sf/542984 SocketServer initializing outbound UDP (2002-04-12) http://python.org/sf/543318 pdb: contradictory msg for "exit"/Ctrl-D (2002-04-14) http://python.org/sf/543674 Closed Patches -------------- new base class for binary packaging (2001-04-10) http://python.org/sf/415226 Solaris pkgtool bdist command (2001-04-10) http://python.org/sf/415227 HP-UX packaging command (2001-04-10) http://python.org/sf/415228 Additions & corrections to libmacui.tex (2001-12-25) http://python.org/sf/496705 Work around for buggy https servers (2002-01-07) http://python.org/sf/500311 string.zfill and unicode (2002-03-28) http://python.org/sf/536241 Remove PyMalloc_* symbols (2002-04-06) http://python.org/sf/540394 From mal@lemburg.com Sun Apr 21 15:31:10 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Sun, 21 Apr 2002 16:31:10 +0200 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objectsunicodeobject.c,2.139,2.140 References: Message-ID: <3CC2CD2E.51BF3F1D@lemburg.com> Tim Peters wrote: > > I expect Martin checked in this change because of the unhappy hours he spent > determining that the previous two versions of this function wrote beyond the > memory they allocated. Since the most recent version still didn't bother to > assert that it wasn't writing out of bounds, I can't blame Martin for > checking in a version that does so assert; since I spent hours on this too, > and this function has a repeated history of bad memory behavior, I viewed > the version Martin replaced as unacceptable. Are you sure, you're talking about the latest version I checked in ? I spent hours on this too and I'm pretty sure to have fixed the buffer overruns now. > However, the slowdown on large strings isn't attractive, and the previous > version could easily enough have asserted its memory correctness. So, why not just add the assert to my original version ? > > -----Original Message----- > > From: python-checkins-admin@python.org > > [mailto:python-checkins-admin@python.org]On Behalf Of M.-A. Lemburg > > Sent: Saturday, April 20, 2002 11:26 AM > > To: loewis@sourceforge.net > > Cc: python-checkins@python.org > > Subject: Re: [Python-checkins] python/dist/src/Objects > > unicodeobject.c,2.139,2.140 > > > > > > loewis@sourceforge.net wrote: > >> > >> Update of /cvsroot/python/python/dist/src/Objects > >> In directory usw-pr-cvs1:/tmp/cvs-serv30961 > >> > >> Modified Files: > >> unicodeobject.c > >> Log Message: > >> Patch #495401: Count number of required bytes for encoding UTF-8 > >> before allocating the target buffer. > > > > Martin, please back out this change again. We have discussed this > > quite a few times and I am against using your strategy since > > it introduces a performance hit which does not relate to the > > gained advantage of (temporarily) using less memory. > > > > Your timings also show this, so I wonder why you checked in this > > patch, e.g. from the patch log: > > """ > > For the current > > CVS (unicodeobject.c 2.136: MAL's change to use a variable > > overalloc), I get > > > > 10 spaces 20.060 > > 100 spaces 2.600 > > 200 spaces 2.030 > > 1000 spaces 0.930 > > 10000 spaces 0.690 > > 10 spaces, 3 bytes 23.520 > > 100 spaces, 3 bytes 3.730 > > 200 spaces, 3 bytes 2.470 > > 1000 spaces, 3 bytes 0.980 > > 10000 spaces, 3 bytes 0.690 > > 30 bytes 24.800 > > 300 bytes 5.220 > > 600 bytes 3.830 > > 3000 bytes 2.480 > > 30000 bytes 2.230 > > > > With unicode3.diff (that's the one you checked in), I get > > > > 10 spaces 19.940 > > 100 spaces 3.260 > > 200 spaces 2.340 > > 1000 spaces 1.650 > > 10000 spaces 1.450 > > 10 spaces, 3 bytes 21.420 > > 100 spaces, 3 bytes 3.410 > > 200 spaces, 3 bytes 2.420 > > 1000 spaces, 3 bytes 1.660 > > 10000 spaces, 3 bytes 1.450 > > 30 bytes 22.260 > > 300 bytes 5.830 > > 600 bytes 4.700 > > 3000 bytes 3.740 > > 30000 bytes 3.540 > > """ > > > > The only case where your patch is faster is for very short > > strings and then only by a few percent, whereas for all > > longer strings you get worse timings, e.g. 3.74 seconds > > compared to 2.48 seconds -- that's a 50% increase in > > run-time ! > > > > Thanks, > > -- > > Marc-Andre Lemburg > > CEO eGenix.com Software GmbH > > ______________________________________________________________________ > > Company & Consulting: http://www.egenix.com/ > > Python Software: http://www.egenix.com/files/python/ > > > > > > _______________________________________________ > > Python-checkins mailing list > > Python-checkins@python.org > > http://mail.python.org/mailman/listinfo/python-checkins -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From nas@python.ca Sun Apr 21 16:49:29 2002 From: nas@python.ca (Neil Schemenauer) Date: Sun, 21 Apr 2002 08:49:29 -0700 Subject: [Python-Dev] test_inspect failure In-Reply-To: ; from tim.one@comcast.net on Sun, Apr 21, 2002 at 01:24:54AM -0400 References: <20020420215407.A12829@glacier.arctrix.com> Message-ID: <20020421084928.A19082@glacier.arctrix.com> Tim Peters wrote: > Do you know whether universal newlines is enabled there FWIW, yes. I suspect the bug only shows up on platforms that use more than one character to terminate lines. Neil From tim.one@comcast.net Sun Apr 21 18:30:48 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 21 Apr 2002 13:30:48 -0400 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Objectsunicodeobject.c,2.139,2.140 In-Reply-To: <3CC2CD2E.51BF3F1D@lemburg.com> Message-ID: [Tim] >> ... behavior, I viewed the version Martin replaced as unacceptable. [M.-A. Lemburg] > Are you sure, you're talking about the latest version I checked in ? Calling the version Martin checked in N, I'm talking about versions N-3, N-2, and N-1. N-3 and N-2 were unacceptable because they wrote out of bounds. N-1 ("the version Martin replaced") was unacceptable because it still didn't assert that it wasn't writing out of bounds. I asked repeatedly in the bug reports opened against N-3 and N-2 that asserts be added. If that had been done in version N-2, at least Barry, Martin, you and I wouldn't have spent additional hours chasing down what turned out to be more out-of-bounds writes (a debug-build run would have triggered an assert directly in the flawed code). > I spent hours on this too and I'm pretty sure to have fixed the > buffer overruns now. You were pretty sure about N-2 too , and the more hours it takes to make tricky code correct, the more suspect that code is. As I most recently implored, in a comment on Barry's bug report against N-2: What I do care about is that there weren't (and still aren't) asserts *verifying* that this delicate code isn't spilling over the allocated bounds. About timing, last time we went around on this, the "measure once, cut once" version of the code was significantly slower in my timing tests too. I don't care so much if the code is tricky, but the trickier the code the more asserts are required. You checked in N-1 (and N-2) without responding to comments like that, and we're all paying for it. You realize asserts go away in the release build, right? They don't cost anything in production mode, they save our ass in debug mode. > ... > So, why not just add the assert to my original version ? I don't know why you didn't . Martin backed out version N, so we're back to N-1, except I see Martin added a crucial assert for you. I added some more since then. From tim.one@comcast.net Sun Apr 21 20:55:48 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 21 Apr 2002 15:55:48 -0400 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Objects unicodeobject.c,2.139,2.140 In-Reply-To: Message-ID: [martin@v.loewis.de] > ... > I found the overallocations strategy that this code had just so > embarrassing: a single three-byte character will cause an > overallocation of three times the memory, which means that the final > realloc will certainly truncate lots of bytes. Well, Python routinely over-allocates string space all over the place, and it doesn't really cost more to throw away a million bytes at the end than to throw away five bytes. That is, the cost of a realloc shrink is at worst proportional to the number of retained bytes; the number of bytes thrown away doesn't matter. For that reason, the thing that puzzles me more about the current state of the code is why it doesn't overallocate by a factor of 4 from the start, and skip all the delicate, repeated "oops! I still didn't get enough memory" logic. It's almost certainly going to do a shrinking realloc at the end anyway. > As a result, we are at the mercy of the realloc implementation here: > If realloc copies memory (such as Pymalloc might some day) when > shrinking buffers, performance will get worse. Since pymalloc only handles small blocks by itself, such copies would be of short contiguous memory regions that are almost certainly in L1 cache (since the encoder function just finished crawling over them). For that reason I doubt the speed hit would be monstrous. > Since this appears to be religious, I'm backing the patch out. It needn't be religious : with sufficient effort we can quantify the tradeoff between the time pymalloc's realloc would take to move shrinking blocks, and the space wasted by realloc leaving them in place. I mentioned before that I want to give pymalloc a new entry point for realloc callers who specifically do or don't want a shrinking realloc to copy memory. For string storage I *suspect* it's better to move the memory (typically exactly one shrinking realloc, after which the string storage is immutable until death), while for most other kinds of storage it's better to leave the block alone (mutable memory that may shrink and grow repeatedly over time). From martin@v.loewis.de Sun Apr 21 22:19:18 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 21 Apr 2002 23:19:18 +0200 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Objects unicodeobject.c,2.139,2.140 In-Reply-To: References: Message-ID: > For string storage I *suspect* it's better to move the memory > (typically exactly one shrinking realloc, after which the string > storage is immutable until death) OTOH, for many string objects, death comes quickly, e.g. if they get concatenated to some other string. It is difficult to anticipate how long any given string will live - wasted memory appears to be best measured in bytes*seconds. Regards, Martin From mal@lemburg.com Sun Apr 21 23:04:41 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 22 Apr 2002 00:04:41 +0200 Subject: [Python-Dev] Re: [Python-checkins]python/dist/src/Objectsunicodeobject.c,2.139,2.140 References: Message-ID: <3CC33779.FAC99B61@lemburg.com> Tim Peters wrote: > > [Tim] > >> ... behavior, I viewed the version Martin replaced as unacceptable. > > [M.-A. Lemburg] > > Are you sure, you're talking about the latest version I checked in ? > > Calling the version Martin checked in N, I'm talking about versions N-3, > N-2, and N-1. N-3 and N-2 were unacceptable because they wrote out of > bounds. N-1 ("the version Martin replaced") was unacceptable because it > still didn't assert that it wasn't writing out of bounds. I asked > repeatedly in the bug reports opened against N-3 and N-2 that asserts be > added. If that had been done in version N-2, at least Barry, Martin, you > and I wouldn't have spent additional hours chasing down what turned out to > be more out-of-bounds writes (a debug-build run would have triggered an > assert directly in the flawed code). Tim, I don't get it... why all the fuzz about some missing asserts ? > > I spent hours on this too and I'm pretty sure to have fixed the > > buffer overruns now. > > You were pretty sure about N-2 too , and the more hours it takes > to make tricky code correct, the more suspect that code is. As I most > recently implored, in a comment on Barry's bug report against N-2: > > What I do care about is that there weren't (and > still aren't) asserts *verifying* that this delicate code > isn't spilling over the allocated bounds. > > About timing, last time we went around on this, > the "measure once, cut once" version of the code was > significantly slower in my timing tests too. I don't care > so much if the code is tricky, but the trickier the code > the more asserts are required. > > You checked in N-1 (and N-2) without responding to comments like that, and > we're all paying for it. You realize asserts go away in the release build, > right? They don't cost anything in production mode, they save our ass in > debug mode. I must have missed Barry post, sorry. I didn't leave out the asserts for any reason -- just didn't think about using them. > > ... > > So, why not just add the assert to my original version ? > > I don't know why you didn't . Martin backed out version N, so we're > back to N-1, except I see Martin added a crucial assert for you. I added > some more since then. Thanks. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From tim.one@comcast.net Mon Apr 22 00:43:44 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 21 Apr 2002 19:43:44 -0400 Subject: [Python-Dev] RE: [Python-checkins]python/dist/src/Objectsunicodeobject.c,2.139,2.140 In-Reply-To: <3CC33779.FAC99B61@lemburg.com> Message-ID: [MAL] > Tim, I don't get it... why all the fuzz about some missing asserts? They were requested repeatedly, and would have saved everyone hours of debugging time. You didn't add asserts when requested, and didn't oppose them either -- they just got ignored. The last two times this routine was implicated in memory corruption, you assumed it was a pymalloc bug, and Martin had to do all the work of indentifying the true cause and then convincing you of that. The first time that was understandable, but the second time it wasn't; then that the third version of the code still didn't try to catch overwrites was over the edge. > I must have missed Barry post, sorry. Barry filed the latest bug report, and you closed it. The memory corruption showed up as a bad UTF-8 translation in Mailman: http://www.python.org/sf/541828 > I didn't leave out the asserts for any reason -- just didn't think > about using them. You would have had you read the comments on the bug reports you closed, so I'm not sure what to make of that. I'll be happy if you use asserts in the future, when the code is getting too tricky to be obviously correct; that shouldn't need to be a protracted battle. From tim.one@comcast.net Mon Apr 22 03:34:45 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 21 Apr 2002 22:34:45 -0400 Subject: [Python-Dev] Breaking builds Message-ID: I may or may not have just broken one or more parts of the build for non-Windows boxes. Please check it out, especially on Linux. What changed: As spelled out before, as part of enabling pymalloc we need to redirect PyMem_{DEL, Del} to the pymalloc free(), for backward compatibility of old extension modules. This creates an internal problem because the parser stuff uses these guys, but has never included Python.h, and can't "see" the declarations of the PyObject_XYZ functions in objimpl.h. Instead, pgenheaders.h has been growing its own maze of includes and defines. I scrapped all that, and had it include Python.h instead, like everything else does. This all works on Windows, but Windows is Special . I at least expect that using pgen to rebuild the grammar files will need to link in obmalloc.o on Linux. Other changes: When PYMALLOC_DEBUG is defined (it is by default in a debug build), *all* PyMem_XYZ functions funnel thru the debugging allocator now. So we get debugging decorations on all allocations then, except for direct calls to platform malloc/realloc/free. Note that platform malloc/realloc/free cannot be mixed with PyMem_XYZ! compile.c actually did that once, and it broke (it's fixed). From Jack.Jansen@cwi.nl Mon Apr 22 09:33:08 2002 From: Jack.Jansen@cwi.nl (Jack Jansen) Date: Mon, 22 Apr 2002 10:33:08 +0200 Subject: [Python-Dev] autoheader warning for WITH_UNIVERSAL_NEWLINES In-Reply-To: Message-ID: <92A7A06E-55CB-11D6-9358-0030655234CE@cwi.nl> On Friday, April 19, 2002, at 10:27 , Martin v. Loewis wrote: > Jack Jansen writes: > >> You're right, so all isn't lost. Now the only prooblem is that the >> order of the defines in pyconfig.h.in has changed again:-( > > Why is that a problem (even if just the only one)? Because I'll be doing a diff of Mac/Include/pyconfig.h with pyconfig.h.in and filling in the correct values for any new defines that have popped up (plus removing ones that have disappeared, etc). So now I'll first have to reorder Mac/Include/pyconfig.h to match pyconfig.h.in. But that should be a once-only issue (although for some reason this "once-only" thing has been happening each year or so, in the past...) -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From barry@zope.com Mon Apr 22 14:22:28 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 22 Apr 2002 09:22:28 -0400 Subject: [Python-Dev] Re: [Python-checkins]python/dist/src/Objectsunicodeobject.c,2.139,2.140 References: <3CC33779.FAC99B61@lemburg.com> Message-ID: <15556.3732.496938.756304@anthem.wooz.org> >>>>> "MAL" == M writes: MAL> I must have missed Barry post, sorry. I didn't leave out the MAL> asserts for any reason -- just didn't think about using them. I think Tim's referring just to the bug report about the regression in s.encode('utf8') that I reported seeing in the email package test. BTW, did you add a separate unittest for this specific failure? -Barry From guido@python.org Mon Apr 22 15:22:41 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 22 Apr 2002 10:22:41 -0400 Subject: [Python-Dev] autoheader warning for WITH_UNIVERSAL_NEWLINES In-Reply-To: Your message of "Mon, 22 Apr 2002 10:33:08 +0200." <92A7A06E-55CB-11D6-9358-0030655234CE@cwi.nl> References: <92A7A06E-55CB-11D6-9358-0030655234CE@cwi.nl> Message-ID: <200204221422.g3MEMfI07306@pcp742651pcs.reston01.va.comcast.net> > So now I'll first have to reorder Mac/Include/pyconfig.h to match > pyconfig.h.in. But that should be a once-only issue (although for some > reason this "once-only" thing has been happening each year or so, in the > past...) Occasionally autoconfig changes its mind about how to order this file. I think I've seen versions disagree about whether sorting was case-sensitive or not, and what to do with leading underscores, or something like that. --Guido van Rossum (home page: http://www.python.org/~guido/) From sjoerd@acm.org Mon Apr 22 15:56:30 2002 From: sjoerd@acm.org (Sjoerd Mullender) Date: Mon, 22 Apr 2002 16:56:30 +0200 Subject: [Python-Dev] autoheader warning for WITH_UNIVERSAL_NEWLINES In-Reply-To: Your message of Mon, 22 Apr 2002 10:22:41 -0400. <200204221422.g3MEMfI07306@pcp742651pcs.reston01.va.comcast.net> References: <92A7A06E-55CB-11D6-9358-0030655234CE@cwi.nl> <200204221422.g3MEMfI07306@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204221456.g3MEuUx16745@indus.ins.cwi.nl> This may well depend on your locale setting. At least on Linux there is a difference in default sort order in the sort program between LANG=en_US and LANG=C. On Mon, Apr 22 2002 Guido van Rossum wrote: > > So now I'll first have to reorder Mac/Include/pyconfig.h to match > > pyconfig.h.in. But that should be a once-only issue (although for some > > reason this "once-only" thing has been happening each year or so, in the > > past...) > > Occasionally autoconfig changes its mind about how to order this > file. I think I've seen versions disagree about whether sorting was > case-sensitive or not, and what to do with leading underscores, or > something like that. > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > -- Sjoerd Mullender From David Abrahams" This appears to be a 2.2.1 bug. Just looking for confirmation or correction before I put it in the bug tracker: Python 2.2 (#28, Mar 13 2002, 23:18:18) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> coerce >>> class ex(object): ... def __init__(self, v): ... self.v = v ... def __coerce__(self, other): ... return self, ex(other) ... def __add__(self, rhs): ... return ex(self.v + rhs.v) ... def __repr__(self): ... return 'ex(' + repr(self.v) + ')' ... >>> ex(3) ex(3) >>> coerce(ex(1),2.3) (ex(1), ex(2.2999999999999998)) >>> ex(1).__add__(ex(2)) ex(3) >>> ex(1)+2.3 Traceback (most recent call last): File "", line 1, in ? File "", line 9, in __add__ AttributeError: 'float' object has no attribute 'v' Thanks, Dave +---------------------------------------------------------------+ David Abrahams C++ Booster (http://www.boost.org) O__ == Pythonista (http://www.python.org) c/ /'_ == resume: http://users.rcn.com/abrahams/resume.html (*) \(*) == email: david.abrahams@rcn.com +---------------------------------------------------------------+ From aahz@pythoncraft.com Mon Apr 22 17:10:37 2002 From: aahz@pythoncraft.com (Aahz) Date: Mon, 22 Apr 2002 12:10:37 -0400 Subject: [Python-Dev] Problems with new-style classes and coercion In-Reply-To: <0bdd01c1ea14$680d9290$6501a8c0@boostconsulting.com> References: <0bdd01c1ea14$680d9290$6501a8c0@boostconsulting.com> Message-ID: <20020422161037.GA7003@panix.com> On Mon, Apr 22, 2002, David Abrahams wrote: > > This appears to be a 2.2.1 bug. Just looking for confirmation or > correction before I put it in the bug tracker: > > Python 2.2 (#28, Mar 13 2002, 23:18:18) [MSC 32 bit (Intel)] on win32 Whatever it is, it's definitely not a 2.2.1 bug, unless the build completely screwed up. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ What if there were no rhetorical questions? From David Abrahams" <20020422161037.GA7003@panix.com> Message-ID: <0c0f01c1ea19$0e55a3a0$6501a8c0@boostconsulting.com> ----- Original Message ----- From: "Aahz" > On Mon, Apr 22, 2002, David Abrahams wrote: > > > > This appears to be a 2.2.1 bug. Just looking for confirmation or > > correction before I put it in the bug tracker: > > > > Python 2.2 (#28, Mar 13 2002, 23:18:18) [MSC 32 bit (Intel)] on win32 > > Whatever it is, it's definitely not a 2.2.1 bug, unless the build > completely screwed up. Could you please clarify? Have I forgotten how __coerce__ is supposed to work? Is what I posted the expected behavior? -Dave From aahz@pythoncraft.com Mon Apr 22 17:22:26 2002 From: aahz@pythoncraft.com (Aahz) Date: Mon, 22 Apr 2002 12:22:26 -0400 Subject: [Python-Dev] Problems with new-style classes and coercion In-Reply-To: <0c0f01c1ea19$0e55a3a0$6501a8c0@boostconsulting.com> References: <0bdd01c1ea14$680d9290$6501a8c0@boostconsulting.com> <20020422161037.GA7003@panix.com> <0c0f01c1ea19$0e55a3a0$6501a8c0@boostconsulting.com> Message-ID: <20020422162226.GA10407@panix.com> On Mon, Apr 22, 2002, David Abrahams wrote: > From: "Aahz" >> On Mon, Apr 22, 2002, David Abrahams wrote: >>> >>> This appears to be a 2.2.1 bug. Just looking for confirmation or >>> correction before I put it in the bug tracker: >>> >>> Python 2.2 (#28, Mar 13 2002, 23:18:18) [MSC 32 bit (Intel)] on win32 >> >> Whatever it is, it's definitely not a 2.2.1 bug, unless the build >> completely screwed up. > > Could you please clarify? Have I forgotten how __coerce__ is supposed to > work? Is what I posted the expected behavior? The header for your interactive session says "2.2" (see quote above) instead of "2.2.1". That means you're *NOT* using 2.2.1, unless the build process for 2.2.1 is completely hosed. Please test against 2.2.1 if you want to report any bugs. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ What if there were no rhetorical questions? From guido@python.org Mon Apr 22 17:47:06 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 22 Apr 2002 12:47:06 -0400 Subject: [Python-Dev] Problems with new-style classes and coercion In-Reply-To: Your message of "Mon, 22 Apr 2002 10:43:08 CDT." <0bdd01c1ea14$680d9290$6501a8c0@boostconsulting.com> References: <0bdd01c1ea14$680d9290$6501a8c0@boostconsulting.com> Message-ID: <200204221647.g3MGl6x09433@pcp742651pcs.reston01.va.comcast.net> > Python 2.2 (#28, Mar 13 2002, 23:18:18) [MSC 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> coerce > > >>> class ex(object): > ... def __init__(self, v): > ... self.v = v > ... def __coerce__(self, other): > ... return self, ex(other) > ... def __add__(self, rhs): > ... return ex(self.v + rhs.v) > ... def __repr__(self): > ... return 'ex(' + repr(self.v) + ')' > ... > >>> ex(3) > ex(3) > >>> coerce(ex(1),2.3) > (ex(1), ex(2.2999999999999998)) > >>> ex(1).__add__(ex(2)) > ex(3) > >>> ex(1)+2.3 > Traceback (most recent call last): > File "", line 1, in ? > File "", line 9, in __add__ > AttributeError: 'float' object has no attribute 'v' New-style classes don't support __coerce__. It's a long and sad story, but basically __coerce__ was a mistake, and I've stopped supporting it for new-style classes. This means that the + operator doesn't call __coerce__. For some reason, coerce() still calls it; maybe that was a mistake. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Mon Apr 22 17:47:10 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 22 Apr 2002 12:47:10 -0400 Subject: [Python-Dev] Problems with new-style classes and coercion In-Reply-To: <20020422162226.GA10407@panix.com> Message-ID: > Python 2.2 (#28, Mar 13 2002, 23:18:18) [MSC 32 bit (Intel)] on win32 FYI, the header line for the distributed PLabs 2.2.1 is Python 2.2.1 (#34, Apr 9 2002, 19:34:33) [MSC 32 bit (Intel)] on win32 From jeremy@zope.com Mon Apr 22 18:30:30 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 22 Apr 2002 13:30:30 -0400 Subject: [Python-Dev] LOAD_NAME & classes Message-ID: <15556.18614.750212.393449@slothrop.zope.com> There's a bug report on SF that notes there is a difference between: def f(x): class Foo: x = x and x = 1 class Foo: x = x The latter works because the class namespace uses LOAD_NAME and finds a global binding for x. The fact that x is also a local is invisible to LOAD_NAME. The former fails because x is none of locals, globals, or builtins, although it is bound in an enclosing scope. LOAD_NAME knows nothing about free variables, however, so it just blows up. Do we want to do anything about this apparent inconsistency? LOAD_NAME is obviously necessary to make stuff like exec work at all, and after a recent bug fix, it evens works as documented for nested scopes. But the docs for a class namespace (there aren't any, right?) don't suggest there is anything special going on. I imagine it would be possible to stop using LOAD_NAME for classes, but I'm not sure if that's a good thing. It could presumably break code that relies on LOAD_NAME's old fashioned search. It also seems like a non trivial amount of work because we'd need a new LOAD/STORE combo that only searched a locals dict. (Maybe that's not so hard.) I think module namespaces also use LOAD_NAME, but it's not clear why. Isn't a module's locals the same as its globals? If so, LOAD_GLOBAL could be used for all names. Any opinion on whether this is worth fixing? And is it a bug fix or a new feature? Jeremy From guido@python.org Mon Apr 22 19:15:51 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 22 Apr 2002 14:15:51 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: Your message of "Mon, 22 Apr 2002 13:30:30 EDT." <15556.18614.750212.393449@slothrop.zope.com> References: <15556.18614.750212.393449@slothrop.zope.com> Message-ID: <200204221815.g3MIFq910133@pcp742651pcs.reston01.va.comcast.net> > There's a bug report on SF that notes there is a difference between: > > def f(x): > class Foo: > x = x > > and > > x = 1 > class Foo: > x = x > > The latter works because the class namespace uses LOAD_NAME and finds > a global binding for x. The fact that x is also a local is invisible > to LOAD_NAME. > > The former fails because x is none of locals, globals, or builtins, > although it is bound in an enclosing scope. LOAD_NAME knows nothing > about free variables, however, so it just blows up. > > Do we want to do anything about this apparent inconsistency? > > LOAD_NAME is obviously necessary to make stuff like exec work at all, > and after a recent bug fix, it evens works as documented for nested > scopes. But the docs for a class namespace (there aren't any, right?) > don't suggest there is anything special going on. > > I imagine it would be possible to stop using LOAD_NAME for classes, > but I'm not sure if that's a good thing. It could presumably break > code that relies on LOAD_NAME's old fashioned search. It also seems > like a non trivial amount of work because we'd need a new LOAD/STORE > combo that only searched a locals dict. (Maybe that's not so hard.) > > I think module namespaces also use LOAD_NAME, but it's not clear why. > Isn't a module's locals the same as its globals? If so, LOAD_GLOBAL > could be used for all names. > > Any opinion on whether this is worth fixing? And is it a bug fix or a > new feature? I just tried this in 2.1 (without nested scopes enabled) and there the first example fails too. While it's slightly confusing, it's consistent with the rule that class bodies don't play the nested scopes game, and I think it shouldn't be fixed. Otherwise you'd have the confusing issue that a function inside a class can't see the class scope, but a class inside a function can see the function scope. Better if neither can see the other. --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@zope.com Mon Apr 22 19:21:45 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 22 Apr 2002 14:21:45 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <200204221815.g3MIFq910133@pcp742651pcs.reston01.va.comcast.net> References: <15556.18614.750212.393449@slothrop.zope.com> <200204221815.g3MIFq910133@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15556.21689.735493.756176@slothrop.zope.com> >>>>> "GvR" == Guido van Rossum writes: > def f(x): > class Foo: > x = x > > and > > x = 1 > class Foo: > x = x > GvR> I just tried this in 2.1 (without nested scopes enabled) and GvR> there the first example fails too. I think you misunderstood my original explanation. With the current implementations, the first example fails and the second example works. I think the bug is that the second example works. GvR> While it's slightly confusing, it's consistent with the rule GvR> that class bodies don't play the nested scopes game, and I GvR> think it shouldn't be fixed. Otherwise you'd have the GvR> confusing issue that a function inside a class can't see the GvR> class scope, but a class inside a function can see the function GvR> scope. Better if neither can see the other. None of the documentation suggests that class and functions don't see each other. Rather, it says the free variables are resolved in the nearest enclosing function scope. The current implementation supports resolution of free variables in class scope, e.g. def f(x): class Foo: print x >>> f(3) 3 I think the problem is with x = x, which ought to be an error if x is a local and x is unbound. The code will succeed, nested or otherwise, if x happens to be a module global. The confusion is that the code will not succeed if x is defined in an enclosing, non-top-level scope. Jeremy From guido@python.org Mon Apr 22 19:39:12 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 22 Apr 2002 14:39:12 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: Your message of "Mon, 22 Apr 2002 14:21:45 EDT." <15556.21689.735493.756176@slothrop.zope.com> References: <15556.18614.750212.393449@slothrop.zope.com> <200204221815.g3MIFq910133@pcp742651pcs.reston01.va.comcast.net> <15556.21689.735493.756176@slothrop.zope.com> Message-ID: <200204221839.g3MIdCE10259@pcp742651pcs.reston01.va.comcast.net> > >>>>> "GvR" == Guido van Rossum writes: > > > def f(x): > > class Foo: > > x = x > > > > and > > > > x = 1 > > class Foo: > > x = x > > GvR> I just tried this in 2.1 (without nested scopes enabled) and > GvR> there the first example fails too. > > I think you misunderstood my original explanation. With the current > implementations, the first example fails and the second example > works. I think the bug is that the second example works. But it has always worked, and I definitely don't want to break this now. I don't understand why you want to break it? That would definitely break working code! > GvR> While it's slightly confusing, it's consistent with the rule > GvR> that class bodies don't play the nested scopes game, and I > GvR> think it shouldn't be fixed. Otherwise you'd have the > GvR> confusing issue that a function inside a class can't see the > GvR> class scope, but a class inside a function can see the function > GvR> scope. Better if neither can see the other. > > None of the documentation suggests that class and functions don't see > each other. Rather, it says the free variables are resolved in the > nearest enclosing function scope. Oops, I misremembered how this works. > The current implementation supports resolution of free variables in > class scope, e.g. > > def f(x): > class Foo: > print x > > >>> f(3) > 3 > > I think the problem is with x = x, which ought to be an error if x is > a local and x is unbound. The code will succeed, nested or otherwise, > if x happens to be a module global. The confusion is that the code > will not succeed if x is defined in an enclosing, non-top-level > scope. Nested scopes in Python will never be completely intuitive, because they break the illusion that "there's only runtime" (as Samuele summarized it a while ago). That said, I can't decide whether it's better to make the first example work, or reject the bug report. I'm tempted to say "the ice is thin here, don't play with fire" or something cryptic like that, and leave it. --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@zope.com Mon Apr 22 19:37:39 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 22 Apr 2002 14:37:39 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <200204221839.g3MIdCE10259@pcp742651pcs.reston01.va.comcast.net> References: <15556.18614.750212.393449@slothrop.zope.com> <200204221815.g3MIFq910133@pcp742651pcs.reston01.va.comcast.net> <15556.21689.735493.756176@slothrop.zope.com> <200204221839.g3MIdCE10259@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15556.22643.641691.447372@slothrop.zope.com> >>>>> "GvR" == Guido van Rossum writes: >> I think you misunderstood my original explanation. With the >> current implementations, the first example fails and the second >> example works. I think the bug is that the second example works. GvR> But it has always worked, and I definitely don't want to break GvR> this now. I don't want to break this now either (although I don't think either of my messages betrayed that intent). I think it's a wart we live with, but I'd love to fix it when we get to Py3K. (Will we ever?) GvR> I don't understand why you want to break it? That would GvR> definitely break working code! And there's the rub. >> I think the problem is with x = x, which ought to be an error if >> x is a local and x is unbound. The code will succeed, nested or >> otherwise, if x happens to be a module global. The confusion is >> that the code will not succeed if x is defined in an enclosing, >> non-top-level scope. GvR> Nested scopes in Python will never be completely intuitive, GvR> because they break the illusion that "there's only runtime" (as GvR> Samuele summarized it a while ago). Wasn't that about macros? GvR> That said, I can't decide whether it's better to make the first GvR> example work, or reject the bug report. I'm tempted to say GvR> "the ice is thin here, don't play with fire" or something GvR> cryptic like that, and leave it. In the bug tracker (532860), I suggested "The result of a reference to an unbound local name in a class block is undefined." With a note that says this is for backwards compatibility and another that explains how CPython is currently implemented. I have a separate bug report about ref man documentation for all this. The ref man documents that world the way I'd like it to be, nicely glossing over all the messy details like LOAD_NAME. I think we need an extra section (4.1.1) that explains some of the messy details. Jeremy From David Abrahams" <200204221647.g3MGl6x09433@pcp742651pcs.reston01.va.comcast.net> Message-ID: <0d8e01c1ea2d$ba8b6510$6501a8c0@boostconsulting.com> ----- Original Message ----- From: "Guido van Rossum" > New-style classes don't support __coerce__. It's a long and sad > story, but basically __coerce__ was a mistake, and I've stopped > supporting it for new-style classes. That is actually a huge relief, as far as I'm concerned. Boost.Python's overloading support should handle heterogeneous operator arguments quite nicely without any intervening __coerce__. > This means that the + operator > doesn't call __coerce__. For some reason, coerce() still calls it; > maybe that was a mistake. Maybe; maybe not. Maybe it would be a good idea to issue a warning when a new-style class defines __coerce__. BTW, thanks to everyone for pointing out my version mismatch problem. I have installed and tested against 2.2.1, but needed to update my paths to invoke it from the command-line. -Dave From aahz@pythoncraft.com Mon Apr 22 19:44:07 2002 From: aahz@pythoncraft.com (Aahz) Date: Mon, 22 Apr 2002 14:44:07 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <200204221839.g3MIdCE10259@pcp742651pcs.reston01.va.comcast.net> References: <15556.18614.750212.393449@slothrop.zope.com> <200204221815.g3MIFq910133@pcp742651pcs.reston01.va.comcast.net> <15556.21689.735493.756176@slothrop.zope.com> <200204221839.g3MIdCE10259@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020422184407.GA25137@panix.com> On Mon, Apr 22, 2002, Guido van Rossum wrote: > > That said, I can't decide whether it's better to make the first > example work, or reject the bug report. I'm tempted to say "the ice > is thin here, don't play with fire" or something cryptic like that, > and leave it. "On thin ice, play with fire you should not." -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ What if there were no rhetorical questions? From guido@python.org Mon Apr 22 19:54:34 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 22 Apr 2002 14:54:34 -0400 Subject: [Python-Dev] Problems with new-style classes and coercion In-Reply-To: Your message of "Mon, 22 Apr 2002 13:44:21 CDT." <0d8e01c1ea2d$ba8b6510$6501a8c0@boostconsulting.com> References: <0bdd01c1ea14$680d9290$6501a8c0@boostconsulting.com> <200204221647.g3MGl6x09433@pcp742651pcs.reston01.va.comcast.net> <0d8e01c1ea2d$ba8b6510$6501a8c0@boostconsulting.com> Message-ID: <200204221854.g3MIsYA10489@pcp742651pcs.reston01.va.comcast.net> > > This means that the + operator > > doesn't call __coerce__. For some reason, coerce() still calls it; > > maybe that was a mistake. > > Maybe; maybe not. Maybe it would be a good idea to issue a warning when > a new-style class defines __coerce__. I think I agree. I'll get to this before 2.3 is released. ;-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 22 20:04:32 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 22 Apr 2002 15:04:32 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: Your message of "Mon, 22 Apr 2002 14:37:39 EDT." <15556.22643.641691.447372@slothrop.zope.com> References: <15556.18614.750212.393449@slothrop.zope.com> <200204221815.g3MIFq910133@pcp742651pcs.reston01.va.comcast.net> <15556.21689.735493.756176@slothrop.zope.com> <200204221839.g3MIdCE10259@pcp742651pcs.reston01.va.comcast.net> <15556.22643.641691.447372@slothrop.zope.com> Message-ID: <200204221904.g3MJ4XF10514@pcp742651pcs.reston01.va.comcast.net> > >>>>> "GvR" == Guido van Rossum writes: > > >> I think you misunderstood my original explanation. With the > >> current implementations, the first example fails and the second > >> example works. I think the bug is that the second example works. > > GvR> But it has always worked, and I definitely don't want to break > GvR> this now. > > I don't want to break this now either (although I don't think either > of my messages betrayed that intent). The fact that you called it a bug (rather than a problem) made me think this. > I think it's a wart we live > with, but I'd love to fix it when we get to Py3K. (Will we ever?) There probably won't ever be an opportunity to start over and be truly incompatible. I don't think I'm up for a herculean effort like the Perl 6 folks are making. > GvR> I don't understand why you want to break it? That would > GvR> definitely break working code! > > And there's the rub. > > >> I think the problem is with x = x, which ought to be an error if > >> x is a local and x is unbound. The code will succeed, nested or > >> otherwise, if x happens to be a module global. The confusion is > >> that the code will not succeed if x is defined in an enclosing, > >> non-top-level scope. > > GvR> Nested scopes in Python will never be completely intuitive, > GvR> because they break the illusion that "there's only runtime" (as > GvR> Samuele summarized it a while ago). > > Wasn't that about macros? That was the context, but it nicely summarizes a lot about Python, and about scripting languages in general. It may even be an explanation of the success of scripting languages: typical programmers probably have a much better mental model of runtime than of compile time. Variable scope already violates the "only runtime" rule, and this has caused endless complaints and misunderstandings. Before nested scopes, I've toyed with the idea of making LOAD_FAST fall through (conceptually) to LOAD_GLOBAL when the local slot is NULL, just to shut up the complaints. Then this example (whose failure always surprises newcomers) would work: x = 10 def f(): print x x = 12 f() But now that we have nested scopes, I don't think this would be feasible -- the fallback would have to dynamically inspect all surrounding scopes. > GvR> That said, I can't decide whether it's better to make the first > GvR> example work, or reject the bug report. I'm tempted to say > GvR> "the ice is thin here, don't play with fire" or something > GvR> cryptic like that, and leave it. > > In the bug tracker (532860), I suggested "The result of a reference to > an unbound local name in a class block is undefined." With a note > that says this is for backwards compatibility and another that > explains how CPython is currently implemented. At least this leaves the door open to fixing it my way. :-) > I have a separate bug report about ref man documentation for all > this. The ref man documents that world the way I'd like it to be, > nicely glossing over all the messy details like LOAD_NAME. I think we > need an extra section (4.1.1) that explains some of the messy details. You can never go wrong by documenting something. (Famous last words, I expect. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@zope.com Mon Apr 22 19:59:48 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 22 Apr 2002 14:59:48 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <20020422184407.GA25137@panix.com> References: <15556.18614.750212.393449@slothrop.zope.com> <200204221815.g3MIFq910133@pcp742651pcs.reston01.va.comcast.net> <15556.21689.735493.756176@slothrop.zope.com> <200204221839.g3MIdCE10259@pcp742651pcs.reston01.va.comcast.net> <20020422184407.GA25137@panix.com> Message-ID: <15556.23972.251117.232019@slothrop.zope.com> >>>>> "A" == aahz writes: A> "On thin ice, play with fire you should not." A> What if there were no rhetorical questions? I think I should read this in a Frank Oz voice, right? Somehow, though, it makes me think: "but sometimes my arms bend back." can-guido-pronounce-his-own-name-ly y'rs, Jeremy From tim.one@comcast.net Mon Apr 22 20:48:24 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 22 Apr 2002 15:48:24 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <200204221904.g3MJ4XF10514@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > ... > Variable scope already violates the "only runtime" rule, and this has > caused endless complaints and misunderstandings. Any scope rules do: you can't win here. You can only choose which specific rules cause endless complaints and misunderstandings. Pure dynamic scoping is the easiest for newcomers to understand, and it's a Bad Idea despite that. > Before nested scopes, I've toyed with the idea of making LOAD_FAST > fall through (conceptually) to LOAD_GLOBAL when the local slot is NULL, > just to shut up the complaints. Then this example (whose failure > always surprises newcomers) It surprises some newcomers, plus Andrew Kuchling . > would work: > > x = 10 > def f(): > print x > x = 12 > f() But this still wouldn't: x = 10 def f(): print x def g(): x = 12 f() g() You can't explain why that doesn't print 12 without insisting that different textual regions have different scopes. Newbie complaints and misunderstandings follow. Under an "only runtime" conception, the only obvious behavior is that x is bound to 12 at the time f() is called, so of course it should print 12. It takes a certain sophistication to understand that the name "x" means different things in different parts of the file. Once that's understood, the behavior of your example is no longer a mystery; but before it's understood, your example is only one of countless confusions. Indeed, the single most frequent newbie confusion I've heard over the last year is variations on "OK, I did 'import math' like you said, but 'sin(x)' *still* gives a NameError!". > But now that we have nested scopes, I don't think this would be > feasible -- the fallback would have to dynamically inspect all > surrounding scopes. As above, dynamic scoping is what newbies expect -- but it would be a great disservice to give it to them. pons-asinorum-ly y'rs - tim From guido@python.org Mon Apr 22 21:11:40 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 22 Apr 2002 16:11:40 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: Your message of "Mon, 22 Apr 2002 15:48:24 EDT." References: Message-ID: <200204222011.g3MKBeu10941@pcp742651pcs.reston01.va.comcast.net> [Guido] > > would work: > > > > x = 10 > > def f(): > > print x > > x = 12 > > f() [Tim] > But this still wouldn't: > > x = 10 > def f(): > print x > > def g(): > x = 12 > f() > > g() > > You can't explain why that doesn't print 12 without insisting that > different textual regions have different scopes. Newbie complaints > and misunderstandings follow. Under an "only runtime" conception, > the only obvious behavior is that x is bound to 12 at the time f() > is called, so of course it should print 12. It takes a certain > sophistication to understand that the name "x" means different > things in different parts of the file. Once that's understood, the > behavior of your example is no longer a mystery; but before it's > understood, your example is only one of countless confusions. I think this is a red herring, and not an argument against what I proposed. The "only runtime" rules doesn't require dynamic scopes (I agree that dynamic scopes would be bad). Dynamic scopes, and your example, mix up the call context with the definition context. My example takes the definition context, and applies the "is x defined in this scope?" test at runtime instead of at compile time. Very different! --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Mon Apr 22 22:23:46 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 22 Apr 2002 17:23:46 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <200204222011.g3MKBeu10941@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > I think this is a red herring, and not an argument against what I > proposed. Sure it does . It's one example of the larger point that *any* scope rules confuse newbies. Lexical scoping doesn't come naturally except to born Schemers, and indeed didn't come naturally to language designers either (e.g., early LISPs all had dynamic scoping). So it goes. > The "only runtime" rules doesn't require dynamic scopes (I agree that > dynamic scopes would be bad). Dynamic scopes, and your example, mix up > the call context with the definition context. My example takes the > definition context, and applies the "is x defined in this scope?" test > at runtime instead of at compile time. Very different! Rewrite it trivially: def f(): print x x = 12 x = 10 # moved the line down f() I don't know what you mean by "definition context" (and neither will a newbie), but any guess I'm likely to come up with wouldn't include the idea that "x = 10" is now in f's definition context. You would have to spell out that there are three namespaces at work here, and that what "x" means in f depends on the dynamic state of two of them, and simply can't be known in general without running the program. If that's not dynamic scoping, it's too close for me to believe a distinction is worth making. BTW, I consider Python's treatment of global vs builtin namespaces dynamic scoping too, and it's nothing but trouble that globals can mask and unmask builtins dynamically. I'd rather make globals and builtins act more like locals now than make locals act more like they're dynamically scoped. BTW2, I see plenty of UnboundLocalErrors in my own code, and some of those have occurred when the same name is also in use as a global. It's always been a logic error due to forgetting to initialize a local, and usually due to "moving code up" in an editor. It sure wouldn't be doing me any favor to let such code silently pick up whatever crap happened to be bound to the same-named global; UnboundLocalError is a fine bug-catcher. From martin@v.loewis.de Mon Apr 22 22:28:54 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 22 Apr 2002 23:28:54 +0200 Subject: [Python-Dev] autoheader warning for WITH_UNIVERSAL_NEWLINES In-Reply-To: <200204221422.g3MEMfI07306@pcp742651pcs.reston01.va.comcast.net> References: <92A7A06E-55CB-11D6-9358-0030655234CE@cwi.nl> <200204221422.g3MEMfI07306@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > Occasionally autoconfig changes its mind about how to order this > file. I think I've seen versions disagree about whether sorting was > case-sensitive or not, and what to do with leading underscores, or > something like that. The most recent change was due to dropping acconfig.h. Before, I believe autoheader tried to preserve the order in acconfig.h, and prepended all automatic entries in alphabetic order. Now that all entries are generated, alphabetic order should be used in the entire file. Regards, Martin From guido@python.org Mon Apr 22 23:00:15 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 22 Apr 2002 18:00:15 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: Your message of "Mon, 22 Apr 2002 17:23:46 EDT." References: Message-ID: <200204222200.g3MM0Fp19010@pcp742651pcs.reston01.va.comcast.net> > > The "only runtime" rules doesn't require dynamic scopes (I agree > > that dynamic scopes would be bad). Dynamic scopes, and your > > example, mix up the call context with the definition context. My > > example takes the definition context, and applies the "is x > > defined in this scope?" test at runtime instead of at compile > > time. Very different! > > Rewrite it trivially: > > def f(): > print x > x = 12 > > x = 10 # moved the line down > f() > > I don't know what you mean by "definition context" (and neither will > a newbie), but any guess I'm likely to come up with wouldn't include > the idea that "x = 10" is now in f's definition context. You would > have to spell out that there are three namespaces at work here, and > that what "x" means in f depends on the dynamic state of two of > them, and simply can't be known in general without running the > program. If that's not dynamic scoping, it's too close for me to > believe a distinction is worth making. I seem to have trouble explaining what I meant. Long ago, before I introduced LOAD_FAST and friends, Python had something that for want of a better term I'll call "lexical scoping with dynamic lookup". It did a dynamic lookup in a (max 3 deep: local / global / builtin) stack of namespaces, but the set of namespaces was determined by the compiler. This does not have the problems of dynamic scoping (the caller's stack frame can't cause trouble). But it also doesn't have the problem of the current strict static scoping. I like the older model better than the current model (apart from nested scopes) and I believe that the "only runtime" rule explains why the old model is more attractive: it doesn't require you to think of the compiler scanning all the code of your function looking for definitions of names. You can think of the interpreter pretty much executing code as it sees it. You have to have a model for name lookup that requires a chaining of namespaces based on where a function is defined, but that's all still purely runtime (it involves executing the def statement). This requires some sophistication for a newbie to understand, but it's been explained successfully for years, and the explanation would be easier without UnboundLocalError. Note that it explains your example above completely: the namespace where f is defined contains a definition of x when f is called, and thus the search stops there. > BTW, I consider Python's treatment of global vs builtin namespaces > dynamic scoping too, and it's nothing but trouble that globals can > mask and unmask builtins dynamically. I'd rather make globals and > builtins act more like locals now than make locals act more like > they're dynamically scoped. Um, that's not what I'd call dynamic scoping. It's dynamic lookup. It's trouble for a compiler that wants to optimize builtins, but the semantic model is nice and simple and easy to explain with the "only runtime" rule. > BTW2, I see plenty of UnboundLocalErrors in my own code, and some of > those have occurred when the same name is also in use as a global. > It's always been a logic error due to forgetting to initialize a > local, and usually due to "moving code up" in an editor. It sure > wouldn't be doing me any favor to let such code silently pick up > whatever crap happened to be bound to the same-named global; > UnboundLocalError is a fine bug-catcher. You definitely have a point there -- like with most irritating errors, the coin has two sides. I don't know which side would waste more time. (When UnboundLocalError was still spelled as NameError, I'd bet on the latter.) --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Mon Apr 22 23:48:30 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 22 Apr 2002 18:48:30 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <200204222200.g3MM0Fp19010@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > I seem to have trouble explaining what I meant. I know, and I confess I'm giving you a hard time. There's a point to that too: uniqueness also imposes costs on newbies and/or newcomers. Across the world of programming languages now, dynamic scoping and lexical scoping are "almost entirely *it*". For example, the Perl spelling of the running example here does work the way you intend, but the explanation in Perl is full-blown dynamic scoping: sub g { print "$x\n"; # prints 12 -- "full-blown dynamic scoping" } sub f { print "$x\n"; # prints 10 local($x) = 12; &g(); } $x = 10; &f(); print "$x\n"; # prints 10 Once you make f print 10, you're on that path as far as anyone coming from any other language can tell at first glance (or even second and third). If you go on to make g print 10 too, it's inexplicable via reference to how any other language works. If there were a huge payback for "being different" here, cool, but the only real payback I see is letting newbies avoid learning how lexical scoping works, and only for a little while. > Long ago, before I introduced LOAD_FAST and friends, Python had > something that for want of a better term I'll call "lexical scoping > with dynamic lookup". I'm old enough to remember this . > It did a dynamic lookup in a (max 3 deep: local / global / builtin) > stack of namespaces, but the set of namespaces was determined by the > compiler. This does not have the problems of dynamic scoping (the > caller's stack frame can't cause trouble). But it also doesn't have > the problem of the current strict static scoping. Nor its advantages, including better error detection, and ease of transferring hard-won knowledge among other lexically scoped languages. > I like the older model better than the current model (apart from > nested scopes) and I believe that the "only runtime" rule explains why > the old model is more attractive: it doesn't require you to think of > the compiler scanning all the code of your function looking for > definitions of names. You can think of the interpreter pretty much > executing code as it sees it. You have to have a model for name > lookup that requires a chaining of namespaces based on where a > function is defined, but that's all still purely runtime (it involves > executing the def statement). > > This requires some sophistication for a newbie to understand, but it's > been explained successfully for years, and the explanation would be > easier without UnboundLocalError. > > Note that it explains your example above completely: the namespace > where f is defined contains a definition of x when f is called, and > thus the search stops there. Does it scale? x = 0 def f(i): if i & 4: x = 10 def g(i): if i & 2: x = 20 def h(i): if i & 1: x = 30 print x h(i) g(i) f(3) I can look at that today and predict with confidence that h() will either print 30 (if and only if i is odd), or raise an exception. This is from purely local analysis of h's body -- it doesn't matter that it's nested, and it's irrelvant what the enclosing functions look like or do. That's a great aid to writing correct code. If the value of x h sees *may* come from h, or from g, or from f, or from the module scope instead, depending on i's specific value at the time f is called, there's a lot more to think about. I could keep local+global straight in pre-1.0 Python, although I never got used to the inability to write nested functions that could refer to each other (perhaps you've forgotten how many times you had to explain that one, and how difficult it was to get across?). Now that Python has full-blown nested scopes, the namespace interactions are potentially much more convoluted, and the "purely local analysis" shortcut made possible by everyone else's notion of lexical scoping becomes correspondingly more valuable. > ... > Um, that's not what I'd call dynamic scoping. It's dynamic lookup. I know -- the problem is that you're the only one in the world making this distinction, and that makes it hard to maintain over time. If it had some killer advantage ... but it doesn't seem to. When Python switched to "strict local" names before 1.0, I don't recall anyone complaining -- if there was a real advantage to dynamic lookup at the local scope, it appeared to have escaped Python's users . I'll grant that it did make exec and "import *" more predictable in corner cases. > It's trouble for a compiler that wants to optimize builtins, but the > semantic model is nice and simple and easy to explain with the "only > runtime" rule. Dynamic scoping is also easy to explain, but it doesn't scale. I'm afraid dynamic lookup doesn't scale either. You should have stuck with Python's original two-level namespace, you know <0.9 wink>. the-builtins-didn't-count-ly y'rs - tim From greg@cosc.canterbury.ac.nz Tue Apr 23 00:12:00 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 23 Apr 2002 11:12:00 +1200 (NZST) Subject: [Python-Dev] autoheader warning for WITH_UNIVERSAL_NEWLINES In-Reply-To: <92A7A06E-55CB-11D6-9358-0030655234CE@cwi.nl> Message-ID: <200204222312.LAA01501@s454.cosc.canterbury.ac.nz> Jack Jansen : > Because I'll be doing a diff of Mac/Include/pyconfig.h with > pyconfig.h.in > > So now I'll first have to reorder Mac/Include/pyconfig.h to match > pyconfig.h.in. Sort them both before diffing them. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Tue Apr 23 01:57:15 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 23 Apr 2002 12:57:15 +1200 (NZST) Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: Message-ID: <200204230057.MAA01517@s454.cosc.canterbury.ac.nz> Tim Peters : > Lexical scoping doesn't come naturally except to > born Schemers, and indeed didn't come naturally to language designers either > (e.g., early LISPs all had dynamic scoping). I doubt whether the early designers of Lisp explicitly designed it to have dynamic scoping. Coming from lambda calculus, they were probably *thinking* of lexical scoping; it's just that the implementation they chose had some unfortunate side effects. So, I conjecture that the *idea* of lexical scoping comes naturally enough, but a correct implementation of it doesn't. :-) By the way, even in dynamically scoped Lisp, there's no equivalent of an UnboundLocalError -- to get a local variable at all, you have to bind some initial value to it. So, using Guido's terminology, early Lisp had dynamic scoping, but not dynamic lookup. Confused enough yet?-) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim.one@comcast.net Tue Apr 23 04:21:02 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 22 Apr 2002 23:21:02 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <200204230057.MAA01517@s454.cosc.canterbury.ac.nz> Message-ID: [Greg Ewing] > I doubt whether the early designers of Lisp explicitly designed > it to have dynamic scoping. Coming from lambda calculus, they > were probably *thinking* of lexical scoping; it's just that > the implementation they chose had some unfortunate side effects. I think McCarthy may be a good source : http://www-formal.stanford.edu/jmc/history/lisp/lisp.html The distinction between dynamic and lexical scoping is clear-cut only in hindsight. As McCarthy relates, Algol 60 later bumped into much the same scoping surprises they did. The treatment of free variables in formal logic is also much clearer in hindsight, and was the source of many difficulties before a proper definition of substitution was crafted. Cute: Curry and Schoenfinkel (independently) invented combinatory logic to get rid of free variables (and the difficulties that come with them) entirely, and decades later combinatory logic overthrew the lambda calculus as the basis for high-performance implementations of pure functional languages. Applying a contemporary understanding of these things retroactively just doesn't fly; whatever clarity we have now was a result of their struggles. > ... > By the way, even in dynamically scoped Lisp, there's no > equivalent of an UnboundLocalError -- to get a local variable > at all, you have to bind some initial value to it. There were certainly NameErrors. All you needed to do was evaluate any old function that referenced a free variable, at a time when no binding for that variable was on the stack. You can't do this in Scheme, but it was dead easy in LISP; it's harder in Common Lisp, but dynamic scoping can still be gotten at there. > So, using Guido's terminology, early Lisp had dynamic scoping, but > not dynamic lookup. Confused enough yet?-) Not as confused as McCarthy turned out to have been . From anthony@interlink.com.au Tue Apr 23 05:12:57 2002 From: anthony@interlink.com.au (Anthony Baxter) Date: Tue, 23 Apr 2002 14:12:57 +1000 Subject: [Python-Dev] SF tracker groups ongoing... Message-ID: <200204230412.g3N4CvD22443@localhost.localdomain> At the moment we have the following tracker groups: Any 3rd Party Feature Request Irreproducible Not a Bug Platform-specific Python 2.1.1 Python 2.1.2 Python 2.2 Python 2.2.1 candidate Python 2.3 Trash Can these be cleaned up a lot? merge Python 2.1.1 and 2.1.2 into Python 2.1.x, merge Python 2.2 and Python 2.2.1 candidate into Python 2.2.x. We'd need to mass-move the existing entries to them first, tho. I believe this needs one of the SF admin users to do this... Also, why do we have Groups like '3rd Party', 'Feature Request', 'Irreproducible', 'Not a Bug', 'Platform-specific', and 'Trash'? Some of these seem like resolutions, Feature Requests have their own tracker. On the subject of trackers, if FRs are merged into the main tracker then 'group: feature request' makes sense. I'd hope that merging them should be a matter of a database update at SF, to change the 'atid' of the existing items... has anyone asked SF if it's possible? also, why is there an empty 'support requests' tracker? shouldn't it be deleted? Anthony From fdrake@acm.org Tue Apr 23 05:39:16 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 23 Apr 2002 00:39:16 -0400 Subject: [Python-Dev] SF tracker groups ongoing... In-Reply-To: <200204230412.g3N4CvD22443@localhost.localdomain> References: <200204230412.g3N4CvD22443@localhost.localdomain> Message-ID: <15556.58740.414783.559868@grendel.zope.com> Anthony Baxter writes: > At the moment we have the following tracker groups: ... > Can these be cleaned up a lot? merge Python 2.1.1 and 2.1.2 into > Python 2.1.x, merge Python 2.2 and Python 2.2.1 candidate into > Python 2.2.x. We'd need to mass-move the existing entries to them > first, tho. I believe this needs one of the SF admin users to do > this... The version-specific groups are all relatively new. I'm not aware of anyone really discussing this or thinking through a policy about these. I don't think we can remove any of these, but we should definately consider using "2.x.y" instead of "2.x.1", "2.x.2", etc. > Also, why do we have Groups like '3rd Party', 'Feature Request', > 'Irreproducible', 'Not a Bug', 'Platform-specific', and 'Trash'? > Some of these seem like resolutions, Feature Requests have their > own tracker. The migration from the "old" SF bug & patch trackers was done poorly (when measured against any particular project); they managed to "merge" different fields from the two old trackers in what they thought was the right way (or maybe it happened by accident), and it didn't seem to turn out very well. The fact that the "Group" and "Resolution" kind of overlap is a result of this. > On the subject of trackers, if FRs are merged into the main > tracker then 'group: feature request' makes sense. I'd hope > that merging them should be a matter of a database update at > SF, to change the 'atid' of the existing items... has anyone > asked SF if it's possible? No idea. Is anyone planning to do such a merge? I think I'm the only one who's commented that the FR tracker is a bad idea. (The default setup for a project on SF is definately too "heavy" for my taste, but they are probably thinking of what makes sense for the "SF On-Site" customers. I'd hope the default setup is configurable, though!) > also, why is there an empty 'support requests' tracker? shouldn't > it be deleted? This was "provided" by SF when they released the current trackers. It's a nuisance, but I don't think they can be deleted, only made "non-public". -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From greg@cosc.canterbury.ac.nz Tue Apr 23 06:26:10 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 23 Apr 2002 17:26:10 +1200 (NZST) Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: Message-ID: <200204230526.RAA01538@s454.cosc.canterbury.ac.nz> Tim Peters : > I think McCarthy may be a good source : > > http://www-formal.stanford.edu/jmc/history/lisp/lisp.html >From there: The difficulty was that when an inner recursion occurred, the value of car[x] wanted was the outer value, but the inner value was actually used. In modern terminology, lexical scoping was wanted, and dynamic scoping was obtained. ... I must confess that I regarded this difficulty as just a bug and expressed confidence that Steve Russell would soon fix it. So it seems I was more or less right -- the semantics they wanted was lexical scoping, effectively, even if they didn't explicitly think about it. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim.one@comcast.net Tue Apr 23 06:35:59 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 23 Apr 2002 01:35:59 -0400 Subject: [Python-Dev] SF tracker groups ongoing... In-Reply-To: <15556.58740.414783.559868@grendel.zope.com> Message-ID: [Fred L. Drake, Jr., on SF Groups] > The version-specific groups are all relatively new. I'm not aware of > anyone really discussing this or thinking through a policy about > these. I don't think we can remove any of these, That's right -- Group and Categories can never be removed, only added. It appears possible to rename them, but we've never tried that, and the admin page says in bold red: It is not recommended that you change the artifact category name because other things are dependent upon it. Whatever that means. > but we should definately consider using "2.x.y" instead of "2.x.1", > "2.x.2", etc. It's good to know which specific version of Python a bug was reported against, and e.g. "2.1.1" is more informative than "2.x.y". Perhaps the unspoken assumption here is that you and/or Anthony want the Group to identify ... what? Something else, I guess . > ... > No idea. Is anyone planning to do such a merge [of Feature Requests > into Bugs]? Not that I know of. > I think I'm the only one who's commented that the FR tracker is a bad > idea. They were split out so the bug page would be more informative: only 20% of feature requests have ever been closed, and nobody looks at them routinely. That is, they're effectively immortal. So if they sat on the bug and/or patch pages, they'd make it harder to find the things that may actually get worked on. SF makes this harder than it should be, because there's no way to say "show me everything that's *not* in such-and-such a specific Group". So, in particular, you can't look at a combined tracker summary page and filter out just the "Feature Request" Group. If SF had better display options, I'd be in all in favor of having one tracker handle everything; as is, I'm afraid I very much like having the feature requests out of sight. From Anthony Baxter Tue Apr 23 06:44:34 2002 From: Anthony Baxter (Anthony Baxter) Date: Tue, 23 Apr 2002 15:44:34 +1000 Subject: [Python-Dev] SF tracker groups ongoing... In-Reply-To: Message from Tim Peters of "Tue, 23 Apr 2002 01:35:59 -0400." Message-ID: <200204230544.g3N5iYZ23710@localhost.localdomain> > > but we should definately consider using "2.x.y" instead of "2.x.1", > > "2.x.2", etc. > > It's good to know which specific version of Python a bug was reported > against, and e.g. "2.1.1" is more informative than "2.x.y". Perhaps the > unspoken assumption here is that you and/or Anthony want the Group to > identify ... what? Something else, I guess . It would be nice to know the version, yes, but I'm more looking for things like 'to what does this bug apply'? That a bug was reported against 2.0.1 is completely irrelevant if it's a new bug which has lurked for years. As it is, with the poor filtering and display options available through SF, I can't say 'show me all outstanding 2.1.x bugs'. > > No idea. Is anyone planning to do such a merge [of Feature Requests > > into Bugs]? > Not that I know of. Given the nastiness of working with the trackers, perhaps this isn't a good idea after all. I don't see how we can get the FRs out of the bug tracker, tho. (I think I'm just spoilt having Richard sitting in the same office. I can ask him to make our roundup do what we want, assuming I can't do it myself...) Anthony -- Anthony Baxter It's never too late to have a happy childhood. From tim.one@comcast.net Tue Apr 23 07:15:34 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 23 Apr 2002 02:15:34 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <200204230526.RAA01538@s454.cosc.canterbury.ac.nz> Message-ID: [Greg Ewing] >> http://www-formal.stanford.edu/jmc/history/lisp/lisp.html > From there: > > The difficulty was that when an inner recursion occurred, the value > of car[x] wanted was the outer value, but the inner value was > actually used. In modern terminology, lexical scoping was wanted, > and dynamic scoping was obtained. > ... > I must confess that I regarded this difficulty as just a bug and > expressed confidence that Steve Russell would soon fix it. > So it seems I was more or less right -- the semantics they > wanted was lexical scoping, effectively, even if they didn't > explicitly think about it. They didn't even have names for it, Greg -- note the "in modern terminology" there. This is McCarthy reflecting on things that happened more than 20 years before he wrote the paper. It was so muddy at the time that this paper was reduced to Unfortunately, time did not permit writing an appendix giving the history of the problem, and the interested reader is referred to (Moses 1970) as a place to start. (David Park tells me that Patrick Fischer also had a hand in developing the FUNARG device). Part of the problem is that the Lisp interpreter was written in Lisp, and contrary to hopes it didn't fully specify the semantics: it turned out that what the Lisp implementation just happened to do for a thing sometimes reflected directly in how the Lisp interpreter emulated that thing, and so studying the interpreter code sheds no light on intent in such cases -- it's a kind of "self-fulfilling algorithm" then. "Metacircular interpreters" fell out of favor for specifying language semantics as a result. Modern attempts to rehabilitate the idea are still stumbling over how to nail the intended scoping semantics; see, e.g. (skip to Conclusions): http://linux.rice.edu/~rahul/hbaker/MetaCircular.html From martin@v.loewis.de Tue Apr 23 07:37:03 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 23 Apr 2002 08:37:03 +0200 Subject: [Python-Dev] SF tracker groups ongoing... In-Reply-To: <200204230412.g3N4CvD22443@localhost.localdomain> References: <200204230412.g3N4CvD22443@localhost.localdomain> Message-ID: Anthony Baxter writes: > Can these be cleaned up a lot? As Tim points out, the answer is "no": you never ever can delete groups in a tracker. Regards, Martin From mwh@python.net Tue Apr 23 09:31:08 2002 From: mwh@python.net (Michael Hudson) Date: 23 Apr 2002 09:31:08 +0100 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: Tim Peters's message of "Mon, 22 Apr 2002 17:23:46 -0400" References: Message-ID: <2mznzuddf7.fsf@starship.python.net> Tim Peters writes: > BTW2, I see plenty of UnboundLocalErrors in my own code, and some of > those have occurred when the same name is also in use as a global. > It's always been a logic error due to forgetting to initialize a > local, and usually due to "moving code up" in an editor. It sure > wouldn't be doing me any favor to let such code silently pick up > whatever crap happened to be bound to the same-named global; > UnboundLocalError is a fine bug-catcher. Wouldn't it be nice if these were flagged at compile time! More work for Jeremy . Cheers, M. -- Strangely enough I saw just such a beast at the grocery store last night. Starbucks sells Javachip. (It's ice cream, but that shouldn't be an obstacle for the Java marketing people.) -- Jeremy Hylton, 29 Apr 1997 From barry@zope.com Tue Apr 23 12:39:52 2002 From: barry@zope.com (Barry A. Warsaw) Date: Tue, 23 Apr 2002 07:39:52 -0400 Subject: [Python-Dev] SF tracker groups ongoing... References: <200204230544.g3N5iYZ23710@localhost.localdomain> Message-ID: <15557.18440.28473.125892@anthem.wooz.org> >>>>> "AB" == Anthony Baxter writes: AB> Given the nastiness of working with the trackers, perhaps this AB> isn't a good idea after all. I don't see how we can get the AB> FRs out of the bug tracker, tho. I believe you can change the "Data type" for a particular entry from "Bugs" to "Feature Requests" and that moves the item to the FR tracker. -Barry From fdrake@acm.org Tue Apr 23 13:32:35 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 23 Apr 2002 08:32:35 -0400 Subject: [Python-Dev] SF tracker groups ongoing... In-Reply-To: <15557.18440.28473.125892@anthem.wooz.org> References: <200204230544.g3N5iYZ23710@localhost.localdomain> <15557.18440.28473.125892@anthem.wooz.org> Message-ID: <15557.21603.878792.999058@grendel.zope.com> Barry A. Warsaw writes: > I believe you can change the "Data type" for a particular entry from > "Bugs" to "Feature Requests" and that moves the item to the FR > tracker. Yep. This would probably wreak havoc with the www.python.org/sf/ redirector's cache since it affects the actual URLs of the tracker entries. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From guido@python.org Tue Apr 23 13:35:24 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 23 Apr 2002 08:35:24 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: Your message of "Mon, 22 Apr 2002 18:48:30 EDT." References: Message-ID: <200204231235.g3NCZOP20027@pcp742651pcs.reston01.va.comcast.net> [Tim] > There's a point to that too: uniqueness also imposes costs on > newbies and/or newcomers. Across the world of programming languages > now, dynamic scoping and lexical scoping are "almost entirely *it*". [...] > Nor its advantages, including better error detection, and ease of > transferring hard-won knowledge among other lexically scoped > languages. But Python *is* unique in that it doesn't require declarations. (I've got to admit that the Perl example saddened me. But then in Perl, local variables are a recent invention. :-) We've found before that this can go against what's common knowledge for other language (e.g. integer division). [...] > Does it scale? > > x = 0 > > def f(i): > if i & 4: > x = 10 > def g(i): > if i & 2: > x = 20 > def h(i): > if i & 1: > x = 30 > print x > h(i) > g(i) > > f(3) > > I can look at that today and predict with confidence that h() will > either print 30 (if and only if i is odd), or raise an exception. > This is from purely local analysis of h's body -- it doesn't matter > that it's nested, and it's irrelvant what the enclosing functions > look like or do. That's a great aid to writing correct code. If > the value of x h sees *may* come from h, or from g, or from f, or > from the module scope instead, depending on i's specific value at > the time f is called, there's a lot more to think about. Yup. But it also requires intricate knowledge of Python's rules, which are different than any other language's rules. You simply can't have a variable declaration inside an if statement in other languages that extends to the entire function body -- either the scope would be limited to the block it's in, or the syntax wouldn't allow it. Python's original semantic model on the other hand, and the model that's still used for globals at the global level, gives a clear explanation: a namespace is implemented as a dictionary, and name lookup searches a pre-set sequence of namespaces until it finds a hit. The lexical scoping rule determines how namespaces are combined. Doing the lookup at runtime is easier to understand than doing it at compile time -- even if the compile version might catch more bugs. But I'm repeating myself; I already said that in my previous message. > I could keep local+global straight in pre-1.0 Python, although I > never got used to the inability to write nested functions that could > refer to each other (perhaps you've forgotten how many times you had > to explain that one, and how difficult it was to get across?). No; apart from you, most people were happy with the rule "nested functions don't work". > Now that Python has full-blown nested scopes, the namespace > interactions are potentially much more convoluted, and the "purely > local analysis" shortcut made possible by everyone else's > notion of lexical scoping becomes correspondingly more valuable. I don't know. Full-blown nested scopes make namespace interactions more convoluted no matter *what* set of rules we pick. An alternative implementation model (with associated subtly different semantics semantics) would have been to create an explicit list of the dicts involved in the name resolution for a particular function invocation; we rejected that model because we wanted this to be (nearly) as fast as locals, so we moved more of the analysis to compile time. But by doing so, we introduced more of a dependency on the programmer's ability to understand what happens at compile time, and that breaks the "only runtime exists" illusion. In PEP 267 Jeremy is exploring how to optimize access to globals *without* changing the rules. The change to LOAD_FAST that I considered before would have optimized access to locals without changing the rules, and I still regret that I didn't think of that when I created LOAD_FAST (even though you disagree): the "only runtime" rule is helpful for a large class of programmers, not only newbies, and I'm not sure that adding more and more cruft from truly compiled languages to Python's *semantics* is a good idea. (Adding compiler technology that doesn't change the rules is fine, of course, if it helps optimizations or better diagnostics.) > > ... > > Um, that's not what I'd call dynamic scoping. It's dynamic lookup. > > I know -- the problem is that you're the only one in the world > making this distinction, and that makes it hard to maintain over > time. You can say that, but that doesn't make it so, and it doesn't convince me. The three-scope was gospel in the Python world, and many people actively disliked adding nested scopes (some still do). > If it had some killer advantage ... but it doesn't seem to. > When Python switched to "strict local" names before 1.0, I don't > recall anyone complaining -- if there was a real advantage to > dynamic lookup at the local scope, it appeared to have escaped > Python's users . I'll grant that it did make exec and "import > *" more predictable in corner cases. Well, we gave them a big reason not to complain: this was the singlemost biggest speedup in Python's history. But the rules were definitely harder to explain, because for the first time we had to explain a second compiler pass. > > It's trouble for a compiler that wants to optimize builtins, but the > > semantic model is nice and simple and easy to explain with the "only > > runtime" rule. > > Dynamic scoping is also easy to explain, but it doesn't scale. I'm > afraid dynamic lookup doesn't scale either. You should have stuck > with Python's original two-level namespace, you know <0.9 wink>. We need more than a single example to decide which rules bites worse for large programs. Deep nesting is not common; long functions are. And there the common annoyance is that a change in line 150 can break the code in line 2 of the function. --Guido van Rossum (home page: http://www.python.org/~guido/) From pobrien@orbtech.com Tue Apr 23 14:51:00 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Tue, 23 Apr 2002 08:51:00 -0500 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <200204231235.g3NCZOP20027@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido van Rossum] > > We need more than a single example to decide which rules bites worse > for large programs. Deep nesting is not common; long functions are. > And there the common annoyance is that a change in line 150 can break > the code in line 2 of the function. I'm not exactly sure what you mean by this. Can you share an example? (Not necessarily 150+ lines long, of course.) Thanks. --- Patrick K. O'Brien Orbtech From guido@python.org Tue Apr 23 15:16:37 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 23 Apr 2002 10:16:37 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: Your message of "Tue, 23 Apr 2002 08:51:00 CDT." References: Message-ID: <200204231416.g3NEGb615386@odiug.zope.com> > [Guido van Rossum] > > We need more than a single example to decide which rules bites worse > > for large programs. Deep nesting is not common; long functions are. > > And there the common annoyance is that a change in line 150 can break > > the code in line 2 of the function. > > I'm not exactly sure what you mean by this. Can you share an > example? (Not necessarily 150+ lines long, of course.) Thanks. It's a classic. Before we had UnboundLocalError (i.e. in 1.5.2 and before) this was a common problem on c.l.py: x = "a global" def f(): print x # user thinks this should print the global # 2000 lines of unrelated code for x in "some sequence": # doesn't realize this overrides x do_something_with(x) Calling f() would raise NameError: x, which caused lots of confusion. We added UnboundLocalError th make it clearer what's going on (so at least the experienced c.l.py users would know right away where the problem was :-), but still requires you to know about something obscure that's going on at compile time (the compiler scanning your entire function for variable definitions). --Guido van Rossum (home page: http://www.python.org/~guido/) From pobrien@orbtech.com Tue Apr 23 16:22:28 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Tue, 23 Apr 2002 10:22:28 -0500 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <200204231416.g3NEGb615386@odiug.zope.com> Message-ID: [Guido van Rossum] > > > And there the common annoyance is that a change in line 150 can break > > > the code in line 2 of the function. > > > > I'm not exactly sure what you mean by this. Can you share an > > example? (Not necessarily 150+ lines long, of course.) Thanks. > > It's a classic. Before we had UnboundLocalError (i.e. in 1.5.2 and > before) this was a common problem on c.l.py: > > x = "a global" > > def f(): > print x # user thinks this should print the global > # 2000 lines of unrelated code > for x in "some sequence": # doesn't realize this overrides x > do_something_with(x) > > Calling f() would raise NameError: x, which caused lots of confusion. > > We added UnboundLocalError th make it clearer what's going on (so at > least the experienced c.l.py users would know right away where the > problem was :-), but still requires you to know about something > obscure that's going on at compile time (the compiler scanning your > entire function for variable definitions). Okay. I think I'm following you, but I want to be certain about the statement that "a change in line 150 can break the code in line 2 of the function." Using your example, the function f() works, but only because of a "fortunate" side effect of sorts. So if the code was later changed to "for y in ..." then f() no longer works. But the example is fundamentally flawed to begin with. Proper code shouldn't have to worry that "a change in line 150 can break the code in line 2 of the function." Right? Or am I still missing something? I've never felt that I needed to know about something obscure going on at compile time in order to write decent Python code. Maybe I'm just being paranoid, but this whole discussion just struck me as odd because I can't recall ever having any problem like this. For the most part Python does exactly what I think it should do. And when it doesn't, I'm usually wrong. --- Patrick K. O'Brien Orbtech From sholden@holdenweb.com Tue Apr 23 16:30:59 2002 From: sholden@holdenweb.com (Steve Holden) Date: Tue, 23 Apr 2002 11:30:59 -0400 Subject: [Python-Dev] LOAD_NAME & classes References: <200204231416.g3NEGb615386@odiug.zope.com> Message-ID: <0fd401c1eadb$def24000$d200000a@holdenweb.com> > > [Guido van Rossum] > > > We need more than a single example to decide which rules bites worse > > > for large programs. Deep nesting is not common; long functions are. > > > And there the common annoyance is that a change in line 150 can break > > > the code in line 2 of the function. > > [Patrick] > > I'm not exactly sure what you mean by this. Can you share an > > example? (Not necessarily 150+ lines long, of course.) Thanks. > [Guido] > It's a classic. Before we had UnboundLocalError (i.e. in 1.5.2 and > before) this was a common problem on c.l.py: > > x = "a global" > > def f(): > print x # user thinks this should print the global > # 2000 lines of unrelated code > for x in "some sequence": # doesn't realize this overrides x > do_something_with(x) > > Calling f() would raise NameError: x, which caused lots of confusion. > > We added UnboundLocalError th make it clearer what's going on (so at > least the experienced c.l.py users would know right away where the > problem was :-), but still requires you to know about something > obscure that's going on at compile time (the compiler scanning your > entire function for variable definitions). > So, the problem is the implicit local declaration that assumed when the compiler detects a binding lower down the function's code than the first reference to it, coupled with their function-wide scope. It is compounded by the fact that although the analysis is performed at compile time the error is only reported at run time. Might it make more sense to issue a warning at compile time to the effect that a variable is being used before it's assigned? How completely are (re)bindings detected by the static analysis? Can you necessarily guarantee that a LOAD_FAST is always looking at a local name? [You'll understand I'm not familiar with the details of code generation]. Seems to me the real problem here is explaining (by the interpreter's behavior, rather than in the documentation ;-) the scope of locals and how a name is determined to be local. It might help beginners if there was some really easy way to get a fuller explanation of an error message. Not sure how best that could be retrofitted, but it's better than code breakage. Unfortunately the scoping rules are now "in the wild", so it's not possible to change things too radically without mucho pain for those who have already relied on them. regards Steve -- home: http://www.holdenweb.com/ Python Web Programming: http://pydish.holdenweb.com/pwp/ From guido@python.org Tue Apr 23 16:33:38 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 23 Apr 2002 11:33:38 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: Your message of "Tue, 23 Apr 2002 10:22:28 CDT." References: Message-ID: <200204231533.g3NFXcs30485@odiug.zope.com> > Okay. I think I'm following you, but I want to be certain about the > statement that "a change in line 150 can break the code in line 2 of the > function." Using your example, the function f() works, but only because of a > "fortunate" side effect of sorts. Why is the fact that this works: x = 12 def f(): print x a "fortunate" side effect? That's how the language works! > So if the code was later changed to "for y > in ..." then f() no longer works. But the example is fundamentally flawed to > begin with. Proper code shouldn't have to worry that "a change in line 150 > can break the code in line 2 of the function." Right? Or am I still missing > something? I think you've got it backwards. My complaint is that if f() above eventually grew 150 lines of unrelated code ending with an unrelated assignment to a local variable x, the breakage would show up at an unexpected point. Except for this one, it's hard to make a change at the *tail* of a function that breaks something at the beginning! > I've never felt that I needed to know about something obscure going > on at compile time in order to write decent Python code. Maybe I'm > just being paranoid, but this whole discussion just struck me as odd > because I can't recall ever having any problem like this. For the > most part Python does exactly what I think it should do. And when it > doesn't, I'm usually wrong. This particular form of breakage was a common error reported on c.l.py and to help at python.org until we added UnboundLocalError to make the diagnostic cleaner. Maybe that's all that's needed; getting a NameError when you see this: x = 12 def f(): print x # <--- NameError raised here! ...150 line of code you didn't think could cause the problem... was very disturbing, causing people to look for places where x was deleted from the global namespace later in the program. --Guido van Rossum (home page: http://www.python.org/~guido/) From aleax@aleax.it Tue Apr 23 16:47:30 2002 From: aleax@aleax.it (Alex Martelli) Date: Tue, 23 Apr 2002 17:47:30 +0200 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <0fd401c1eadb$def24000$d200000a@holdenweb.com> References: <200204231416.g3NEGb615386@odiug.zope.com> <0fd401c1eadb$def24000$d200000a@holdenweb.com> Message-ID: On Tuesday 23 April 2002 05:30 pm, Steve Holden wrote: ... > Might it make more sense to issue a warning at compile time to the effect > that a variable is being used before it's assigned? How completely are Hard to make sure, so, it would always be "MIGHT be used before assignment". E.g., def f(): for i in range(6): if i>0: print x, x = str(i) this is OK... no UnboundNameError. Now, change the if's guard to if SolveTuringHaltingProblem(i): print x, to see why it's hard to make sure about this at compile time. A warning might still be OK of course, even though it may need to be worded in wishy-washy "might" terms -- this IS peculiar usage. Alex From guido@python.org Tue Apr 23 16:57:16 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 23 Apr 2002 11:57:16 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: Your message of "Tue, 23 Apr 2002 11:30:59 EDT." <0fd401c1eadb$def24000$d200000a@holdenweb.com> References: <200204231416.g3NEGb615386@odiug.zope.com> <0fd401c1eadb$def24000$d200000a@holdenweb.com> Message-ID: <200204231557.g3NFvGO30632@odiug.zope.com> [Steve Holden] > So, the problem is the implicit local declaration that assumed when > the compiler detects a binding lower down the function's code than > the first reference to it, coupled with their function-wide > scope. It is compounded by the fact that although the analysis is > performed at compile time the error is only reported at run time. Well put. > Might it make more sense to issue a warning at compile time to the > effect that a variable is being used before it's assigned? Yes. Historically (when LOAD_FAST was added) we didn't have a framework for compile-time warnings, but now we do. > How completely are (re)bindings detected by the static analysis? Can > you necessarily guarantee that a LOAD_FAST is always looking at a > local name? [You'll understand I'm not familiar with the details of > code generation]. Yes, LOAD_FAST is only emitted if the compiler has decided that it's a local name, and then there's no exception (though that is exactly what I contemplated changing earlier in this thread). > Seems to me the real problem here is explaining (by the > interpreter's behavior, rather than in the documentation ;-) the > scope of locals and how a name is determined to be local. It might > help beginners if there was some really easy way to get a fuller > explanation of an error message. Not sure how best that could be > retrofitted, but it's better than code breakage. I think if they look up UnboundLocalError in the docs they find the right clues. Of course it's not necessarily easy to find that bit of documentation (though Google's first hit is very helpful). > Unfortunately the scoping rules are now "in the wild", so it's not > possible to change things too radically without mucho pain for those > who have already relied on them. Changing LOAD_FAST so that x = 1 def f(): print x x = 2 print x prints 1 followed by 2 instead of raising UnboundLocalError would not break existing code (except code relying on specific exceptions to be raised -- but such code is always exempt from non-breakage guarantees). --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Tue Apr 23 17:04:21 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 23 Apr 2002 12:04:21 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <200204231235.g3NCZOP20027@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > ... > No; apart from you, most people were happy with the rule "nested > functions don't work". Most, yes, but most people are happy with UnboundLocalError too. The business about nested functions not being able to see each other was a FAQ regardless, and Aaron Watters wrote the FAQ Wizard blurb about it. This FAQ entry was distinct from the FAQ entry about nested functions not being able to see the locals in enclosing functions, despite that they were really the same issue, because people who understood the latter just didn't see the connection to the former without additional words. So the answer to my original question: (perhaps you've forgotten how many times you had to explain that one, and how difficult it was to get across?). is either "yes", or "no, but I eventually left such explaining to others, and so have nothing to forget" . From guido@python.org Tue Apr 23 17:02:28 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 23 Apr 2002 12:02:28 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: Your message of "Tue, 23 Apr 2002 17:47:30 +0200." References: <200204231416.g3NEGb615386@odiug.zope.com> <0fd401c1eadb$def24000$d200000a@holdenweb.com> Message-ID: <200204231602.g3NG2Sr30676@odiug.zope.com> > > Might it make more sense to issue a warning at compile time to the effect > > that a variable is being used before it's assigned? How completely are > > Hard to make sure, so, it would always be "MIGHT be used before > assignment". E.g., > > def f(): > for i in range(6): > if i>0: print x, > x = str(i) > > this is OK... no UnboundNameError. Now, change the if's guard to > if SolveTuringHaltingProblem(i): print x, > to see why it's hard to make sure about this at compile time. > > A warning might still be OK of course, even though it may need to > be worded in wishy-washy "might" terms -- this IS peculiar usage. GCC warns about this stuff as soon as there's a path through the code that could leave a variable unassigned, even if the path may not be reachable depending on what other functions do, and while it occasionally caused my to insert a useless initialization, I've usually found that it was hard enough to prove to myself that the initialization was unneeded to make me happy to insert it. --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@zope.com Tue Apr 23 17:06:37 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Tue, 23 Apr 2002 12:06:37 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <200204231557.g3NFvGO30632@odiug.zope.com> References: <200204231416.g3NEGb615386@odiug.zope.com> <0fd401c1eadb$def24000$d200000a@holdenweb.com> <200204231557.g3NFvGO30632@odiug.zope.com> Message-ID: <15557.34445.141384.490418@slothrop.zope.com> >>>>> "GvR" == Guido van Rossum writes: GvR> Changing LOAD_FAST so that GvR> x = 1 GvR> def f(): GvR> print x GvR> x = 2 GvR> print x GvR> prints 1 followed by 2 instead of raising UnboundLocalError GvR> would not break existing code (except code relying on specific GvR> exceptions to be raised -- but such code is always exempt from GvR> non-breakage guarantees). It wouldn't break existing code, but it would encourage the existence of broken code. It's asking for trouble to write a function that uses a local and a global with the same name. I don't think we should bend the rules to accommodate this kind of confusion. Jeremy From pobrien@orbtech.com Tue Apr 23 17:18:52 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Tue, 23 Apr 2002 11:18:52 -0500 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <200204231533.g3NFXcs30485@odiug.zope.com> Message-ID: [Guido van Rossum] > Why is the fact that this works: > > x = 12 > def f(): > print x > > a "fortunate" side effect? That's how the language works! Yes. I got a fresh cup of coffee and smashed my head against a wall and now my mental processes appear to be working again. At least, there's more squeaking and rattling. My previous choice of words now appears to be rather... unfortunate. > I think you've got it backwards. My complaint is that if f() above > eventually grew 150 lines of unrelated code ending with an unrelated > assignment to a local variable x, the breakage would show up at an > unexpected point. Except for this one, it's hard to make a change at > the *tail* of a function that breaks something at the beginning! Yes, I don't know what I was thinking. Maybe I need a donut. Thanks. --- Patrick K. O'Brien Orbtech From tim.one@comcast.net Tue Apr 23 17:34:48 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 23 Apr 2002 12:34:48 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <2mznzuddf7.fsf@starship.python.net> Message-ID: [Michael Hudson, on UnboundLocalErrors] > ... > Wouldn't it be nice if these were flagged at compile time! More work > for Jeremy . As will (or already has ) been pointed out, the compiler can only guess, and can't always guess right (it could err on the side of bothering you without cause, though). PyChecker goes a long way already. Java has a mass of "definite assignment" rules instead, which in effect decree that you must write code in such a way that the compiler can always guess right. From tim.one@comcast.net Tue Apr 23 17:43:21 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 23 Apr 2002 12:43:21 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <200204231235.g3NCZOP20027@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > ... > But Python *is* unique in that it doesn't require declarations. (I've > got to admit that the Perl example saddened me. But then in Perl, > local variables are a recent invention. :-) Just FYI, "local()" in Perl is very old, and has always used dynamic scoping rules. If you don't wrap local() around a vrbl decl, you get a global vrbl instead, and that's also very old. What's relatively new is the "my()" vrbl wrapper, which asks for a lexically scoped vrbl. What's very new is "our()", which I still haven't figured out: An "our" declares the listed variables to be valid globals within the enclosing block, file, or eval. That is, it has the same scoping rules as a "my" declaration, but does not create a local variable. If more than one value is listed, the list must be placed in parentheses. The our declaration has no semantic effect unless "use strict vars" is in effect, in which case it lets you use the declared global variable without qualifying it with a package name. (But only within the lexical scope of the our declaration. In this it differs from "use vars", which is package scoped.) Anyone want to make a motion that we not ape Perl's scoping rules ? From aahz@pythoncraft.com Tue Apr 23 18:04:25 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 23 Apr 2002 13:04:25 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <200204231557.g3NFvGO30632@odiug.zope.com> References: <200204231416.g3NEGb615386@odiug.zope.com> <0fd401c1eadb$def24000$d200000a@holdenweb.com> <200204231557.g3NFvGO30632@odiug.zope.com> Message-ID: <20020423170424.GA22124@panix.com> On Tue, Apr 23, 2002, Guido van Rossum wrote: > > Changing LOAD_FAST so that > > x = 1 > def f(): > print x > x = 2 > print x > > prints 1 followed by 2 instead of raising UnboundLocalError would not > break existing code (except code relying on specific exceptions to be > raised -- but such code is always exempt from non-breakage > guarantees). -1 I agree with Jeremy that such code is by definition broken because it mixes global and local usage of the same variable within the same scope. That's a nightmare for visual inspection. BTW, the indentation came out bizarrely because you used a TAB char for indentation. Naughty, naughty. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ What if there were no rhetorical questions? From mgilfix@eecs.tufts.edu Tue Apr 23 18:05:14 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Tue, 23 Apr 2002 13:05:14 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: ; from tim.one@comcast.net on Tue, Apr 23, 2002 at 12:43:21PM -0400 References: <200204231235.g3NCZOP20027@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020423130514.E10411@eecs.tufts.edu> On Tue, Apr 23 @ 12:43, Tim Peters wrote: > Just FYI, "local()" in Perl is very old, and has always used dynamic scoping > rules. If you don't wrap local() around a vrbl decl, you get a global vrbl > instead, and that's also very old. What's relatively new is the "my()" vrbl > wrapper, which asks for a lexically scoped vrbl. What's very new is > "our()", which I still haven't figured out: > > An "our" declares the listed variables to be valid globals within > the enclosing block, file, or eval. That is, it has the same scoping > rules as a "my" declaration, but does not create a local variable. If > more than one value is listed, the list must be placed in parentheses. > The our declaration has no semantic effect unless "use strict vars" > is in effect, in which case it lets you use the declared global > variable without qualifying it with a package name. (But only within > the lexical scope of the our declaration. In this it differs from > "use vars", which is package scoped.) > > Anyone want to make a motion that we not ape Perl's scoping rules ? The our is akin to declaring something static in C. Except in Perl, it can apply to an enclosure as well since you can build funcs dynamically. Yay. I think python's scoping rules work just fine :) Has there ever been a discussion about some easy or straight-forward way of sharing a global instance across modules? For example, in a gui app, you might want to structure the program such that there's a global instance app (main.app) of the application that other modules might want to query. I was never to happy with importing main and then using main.app... I felt like I wanted to qualify it as a global, like: from main import global app -- Mike -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From pinard@iro.umontreal.ca Tue Apr 23 18:27:59 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 23 Apr 2002 13:27:59 -0400 Subject: [Python-Dev] Re: LOAD_NAME & classes In-Reply-To: <200204231235.g3NCZOP20027@pcp742651pcs.reston01.va.comcast.net> References: <200204231235.g3NCZOP20027@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido van Rossum] > But Python *is* unique in that it doesn't require declarations. (I've > got to admit that the Perl example saddened me. But then in Perl, > local variables are a recent invention. :-) Guido, I miss the point of the second sentence. Why were you saddened? Don't you prefer _not_ having to declare locals, as Python does now? -- François Pinard http://www.iro.umontreal.ca/~pinard From aahz@pythoncraft.com Tue Apr 23 19:13:14 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 23 Apr 2002 14:13:14 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <20020423130514.E10411@eecs.tufts.edu> References: <200204231235.g3NCZOP20027@pcp742651pcs.reston01.va.comcast.net> <20020423130514.E10411@eecs.tufts.edu> Message-ID: <20020423181314.GA6816@panix.com> On Tue, Apr 23, 2002, Michael Gilfix wrote: > > Has there ever been a discussion about some easy or straight-forward > way of sharing a global instance across modules? For example, in a > gui app, you might want to structure the program such that there's a > global instance app (main.app) of the application that other modules > might want to query. I was never to happy with importing main and then > using main.app... I felt like I wanted to qualify it as a global, like: > > from main import global app The simple way to do it, IMO, is to import a joint module, called something like cfg. I just did a little hack in a script with cfg = imp.new_module('cfg') but I don't know how well that works across modules. I think you'd need to poke it into sys.modules in order for it to be shared, but I haven't tested it. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ What if there were no rhetorical questions? From guido@python.org Tue Apr 23 19:37:59 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 23 Apr 2002 14:37:59 -0400 Subject: [Python-Dev] Re: LOAD_NAME & classes In-Reply-To: Your message of "23 Apr 2002 13:27:59 EDT." References: <200204231235.g3NCZOP20027@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204231837.g3NIbxK31301@odiug.zope.com> > [Guido van Rossum] > > But Python *is* unique in that it doesn't require declarations. (I've > > got to admit that the Perl example saddened me. But then in Perl, > > local variables are a recent invention. :-) > > Guido, I miss the point of the second sentence. Why were you saddened? > Don't you prefer _not_ having to declare locals, as Python does now? > > François Pinard http://www.iro.umontreal.ca/~pinard François, I was sad because Perl seemed to use dynamic scoping when you declare a local variable. Tim has since explained that Perl's local is in fact ancient. I'm also sad that Perl doesn't default to locals, but that's not my problem. --Guido van Rossum (home page: http://www.python.org/~guido/) From martin@v.loewis.de Tue Apr 23 19:35:43 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 23 Apr 2002 20:35:43 +0200 Subject: [Python-Dev] SF tracker groups ongoing... In-Reply-To: <15557.21603.878792.999058@grendel.zope.com> References: <200204230544.g3N5iYZ23710@localhost.localdomain> <15557.18440.28473.125892@anthem.wooz.org> <15557.21603.878792.999058@grendel.zope.com> Message-ID: "Fred L. Drake, Jr." writes: > Yep. This would probably wreak havoc with the www.python.org/sf/ > redirector's cache since it affects the actual URLs of the tracker > entries. That shouldn't stop you from doing it. I'm planning to regenerate the cache on a regular basis (through cron). This isn't implemented yet, but when it is, few people should ever notice problems. Regards, Martin From fdrake@acm.org Tue Apr 23 19:50:41 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 23 Apr 2002 14:50:41 -0400 Subject: [Python-Dev] SF tracker groups ongoing... In-Reply-To: References: <200204230544.g3N5iYZ23710@localhost.localdomain> <15557.18440.28473.125892@anthem.wooz.org> <15557.21603.878792.999058@grendel.zope.com> Message-ID: <15557.44289.530834.260327@grendel.zope.com> Martin v. Loewis writes: > That shouldn't stop you from doing it. I'm planning to regenerate the > cache on a regular basis (through cron). This isn't implemented yet, > but when it is, few people should ever notice problems. Barry & I took a quick look at the redirector earlier. It looks like just removing the cache occaisionally should be sufficient. The more I think about the other alternatives we discussed, the more I don't think it's worth it. Perhaps we should think about the number of hits the redirector gets. Is it really large enough to bother with re-populating the pre-computed list. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From mgilfix@eecs.tufts.edu Tue Apr 23 20:04:57 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Tue, 23 Apr 2002 15:04:57 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <20020423181314.GA6816@panix.com>; from aahz@pythoncraft.com on Tue, Apr 23, 2002 at 02:13:14PM -0400 References: <200204231235.g3NCZOP20027@pcp742651pcs.reston01.va.comcast.net> <20020423130514.E10411@eecs.tufts.edu> <20020423181314.GA6816@panix.com> Message-ID: <20020423150457.A28052@eecs.tufts.edu> On Tue, Apr 23 @ 14:13, Aahz wrote: > On Tue, Apr 23, 2002, Michael Gilfix wrote: > > > > Has there ever been a discussion about some easy or straight-forward > > way of sharing a global instance across modules? For example, in a > > gui app, you might want to structure the program such that there's a > > global instance app (main.app) of the application that other modules > > might want to query. I was never to happy with importing main and then > > using main.app... I felt like I wanted to qualify it as a global, like: > > > > from main import global app > > The simple way to do it, IMO, is to import a joint module, called > something like cfg. I just did a little hack in a script with > > cfg = imp.new_module('cfg') > > but I don't know how well that works across modules. I think you'd need > to poke it into sys.modules in order for it to be shared, but I haven't > tested it. Yeah, that's another good way to do it. Have some sort of configuration object you could query. It would be nice if python offered a facility for sharing pooled data: maybe a global configuration object? That way it's more explicit. It could also remove some redundant code that I tend to put in many applications. import config my_opt = config.get_opt ('mymodule') # Set a new global opt config.set_opt ('attrib') = blah Maybe even provide some hooks for populating the config object. -- Mike -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From pinard@iro.umontreal.ca Tue Apr 23 20:27:37 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 23 Apr 2002 15:27:37 -0400 Subject: [Python-Dev] Re: LOAD_NAME & classes In-Reply-To: <200204231837.g3NIbxK31301@odiug.zope.com> References: <200204231235.g3NCZOP20027@pcp742651pcs.reston01.va.comcast.net> <200204231837.g3NIbxK31301@odiug.zope.com> Message-ID: [Guido van Rossum] > I was sad because Perl seemed to use dynamic scoping when you declare a > local variable. Tim has since explained that Perl's local is in fact > ancient. I'm also sad that Perl doesn't default to locals, but that's > not my problem. Agreed. I've been using Perl since Perl 1, all the way to Perl 5. When the `local()' declaration appeared, probably around Perl 2 or Perl 3 (I do not remember), I started using it everywhere in an effort of protecting each procedure against each other. It was also a convenient way to give names to formal arguments, through "local(name1, name2, ...) = @_" as the first statement in a `sub'. When "my(LIST)" appeared, one or two major releases later, I progressively converted all my programs to use it, as it is more in the spirit of what I wanted. The overall impression that was left to me is that a lot of declarations are needed, anyway, if you consistently aim clean Perl scripts. So, there is now an "our(...)" declaration in Perl? Hopefully, it will not miss me much! :-) And who remembers the "own" specifier of Algol-60? :-) -- François Pinard http://www.iro.umontreal.ca/~pinard From pinard@iro.umontreal.ca Tue Apr 23 20:49:47 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 23 Apr 2002 15:49:47 -0400 Subject: [Python-Dev] Re: LOAD_NAME & classes In-Reply-To: <20020423150457.A28052@eecs.tufts.edu> References: <200204231235.g3NCZOP20027@pcp742651pcs.reston01.va.comcast.net> <20020423130514.E10411@eecs.tufts.edu> <20020423181314.GA6816@panix.com> <20020423150457.A28052@eecs.tufts.edu> Message-ID: [Michael Gilfix] > It would be nice if python offered a facility for sharing pooled data: > maybe a global configuration object? That way it's more explicit. > It could also remove some redundant code that I tend to put in many > applications. > import config > my_opt = config.get_opt ('mymodule') > # Set a new global opt > config.set_opt ('attrib') = blah > Maybe even provide some hooks for populating the config object. I already use `config' (that precise identifier) a lot in my things, so please do not even _think_ using it for something else! :-) :-) For small projects, such a thing might be overkill. For big projects, there are a few (maybe many) avenues and ways how to achieve this sharing, and I'm glad having the freedom of installing subtle nuances between projects, as needed. That is, I would fear a bit being pulled in an arbitrary direction: it would have to be convincingly damned good. :-) Better, maybe, would be to merely document some of the possible avenues. Most of these require only a tiny investment in lines of code anyway. -- François Pinard http://www.iro.umontreal.ca/~pinard From tim.one@comcast.net Tue Apr 23 20:55:09 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 23 Apr 2002 15:55:09 -0400 Subject: [Python-Dev] SF tracker groups ongoing... In-Reply-To: <15557.18440.28473.125892@anthem.wooz.org> Message-ID: [Barry] > I believe you can change the "Data type" for a particular entry from > "Bugs" to "Feature Requests" and that moves the item to the FR > tracker. Yes, but you lose the metadata, like Priority and artifact ID. That's not a killer, just something to be aware of. From mgilfix@eecs.tufts.edu Tue Apr 23 20:59:22 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Tue, 23 Apr 2002 15:59:22 -0400 Subject: [Python-Dev] Re: LOAD_NAME & classes In-Reply-To: ; from pinard@iro.umontreal.ca on Tue, Apr 23, 2002 at 03:49:47PM -0400 References: <200204231235.g3NCZOP20027@pcp742651pcs.reston01.va.comcast.net> <20020423130514.E10411@eecs.tufts.edu> <20020423181314.GA6816@panix.com> <20020423150457.A28052@eecs.tufts.edu> Message-ID: <20020423155921.A142@eecs.tufts.edu> Upon futher reflection, I would agree with you. It's something I find slightly confusing when I first started with python - what the best method sharing a global instance is. I agree any such mechanism needs to be well thought out though. Perhaps some good documentation is the best solution. -- Mike On Tue, Apr 23 @ 15:49, François Pinard wrote: > I already use `config' (that precise identifier) a lot in my things, > so please do not even _think_ using it for something else! :-) :-) > > For small projects, such a thing might be overkill. For big projects, > there are a few (maybe many) avenues and ways how to achieve this sharing, > and I'm glad having the freedom of installing subtle nuances between > projects, as needed. That is, I would fear a bit being pulled in an > arbitrary direction: it would have to be convincingly damned good. :-) > > Better, maybe, would be to merely document some of the possible avenues. > Most of these require only a tiny investment in lines of code anyway. -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From fredrik@pythonware.com Tue Apr 23 21:17:19 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Tue, 23 Apr 2002 22:17:19 +0200 Subject: [Python-Dev] Re: LOAD_NAME & classes References: <200204231235.g3NCZOP20027@pcp742651pcs.reston01.va.comcast.net> <20020423130514.E10411@eecs.tufts.edu> <20020423181314.GA6816@panix.com> <20020423150457.A28052@eecs.tufts.edu> <20020423155921.A142@eecs.tufts.edu> Message-ID: <031b01c1eb03$e012eb10$ced241d5@hagrid> michael wrote: > Upon futher reflection, I would agree with you. It's something I > find slightly confusing when I first started with python - what the > best method sharing a global instance is. I agree any such mechanism > needs to be well thought out though. Perhaps some good documentation > is the best solution. http://www.python.org/doc/current/tut/node8.html#SECTION008100000000000000000 From aahz@pythoncraft.com Tue Apr 23 21:57:12 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 23 Apr 2002 16:57:12 -0400 Subject: [Python-Dev] Re: LOAD_NAME & classes In-Reply-To: <031b01c1eb03$e012eb10$ced241d5@hagrid> References: <200204231235.g3NCZOP20027@pcp742651pcs.reston01.va.comcast.net> <20020423130514.E10411@eecs.tufts.edu> <20020423181314.GA6816@panix.com> <20020423150457.A28052@eecs.tufts.edu> <20020423155921.A142@eecs.tufts.edu> <031b01c1eb03$e012eb10$ced241d5@hagrid> Message-ID: <20020423205711.GA11691@panix.com> On Tue, Apr 23, 2002, Fredrik Lundh wrote: > michael wrote: >> >> Upon futher reflection, I would agree with you. It's something I >> find slightly confusing when I first started with python - what the >> best method sharing a global instance is. I agree any such mechanism >> needs to be well thought out though. Perhaps some good documentation >> is the best solution. > > http://www.python.org/doc/current/tut/node8.html#SECTION008100000000000000000 That addresses the technical mechanism, but not the use case at hand. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ What if there were no rhetorical questions? From fredrik@pythonware.com Tue Apr 23 22:24:44 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Tue, 23 Apr 2002 23:24:44 +0200 Subject: [Python-Dev] Re: LOAD_NAME & classes References: <200204231235.g3NCZOP20027@pcp742651pcs.reston01.va.comcast.net> <20020423130514.E10411@eecs.tufts.edu> <20020423181314.GA6816@panix.com> <20020423150457.A28052@eecs.tufts.edu> <20020423155921.A142@eecs.tufts.edu> <031b01c1eb03$e012eb10$ced241d5@hagrid> <20020423205711.GA11691@panix.com> Message-ID: <001201c1eb0d$4adbde30$ced241d5@hagrid> Aahz wrote: > > http://www.python.org/doc/current/tut/node8.html#SECTION008100000000000000000 > > That addresses the technical mechanism, but not the use case at hand. the use case was "some easy or straight-forward way of sharing a global instance across modules". the answer is "put it in a module". end of story. followups to comp.lang.python, thank you. From aahz@pythoncraft.com Tue Apr 23 22:39:43 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 23 Apr 2002 17:39:43 -0400 Subject: [Python-Dev] Re: LOAD_NAME & classes In-Reply-To: <001201c1eb0d$4adbde30$ced241d5@hagrid> References: <200204231235.g3NCZOP20027@pcp742651pcs.reston01.va.comcast.net> <20020423130514.E10411@eecs.tufts.edu> <20020423181314.GA6816@panix.com> <20020423150457.A28052@eecs.tufts.edu> <20020423155921.A142@eecs.tufts.edu> <031b01c1eb03$e012eb10$ced241d5@hagrid> <20020423205711.GA11691@panix.com> <001201c1eb0d$4adbde30$ced241d5@hagrid> Message-ID: <20020423213943.GA18774@panix.com> On Tue, Apr 23, 2002, Fredrik Lundh wrote: > Aahz wrote: > >>> http://www.python.org/doc/current/tut/node8.html#SECTION008100000000000000000 >> >> That addresses the technical mechanism, but not the use case at hand. > > the use case was "some easy or straight-forward way of > sharing a global instance across modules". > > the answer is "put it in a module". > end of story. That's what I said earlier. But where is this documented? Never mind, I just submitted a FAQ entry, 4.105. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ What if there were no rhetorical questions? From guido@python.org Tue Apr 23 22:45:04 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 23 Apr 2002 17:45:04 -0400 Subject: [Python-Dev] Re: LOAD_NAME & classes In-Reply-To: Your message of "Tue, 23 Apr 2002 17:39:43 EDT." <20020423213943.GA18774@panix.com> References: <200204231235.g3NCZOP20027@pcp742651pcs.reston01.va.comcast.net> <20020423130514.E10411@eecs.tufts.edu> <20020423181314.GA6816@panix.com> <20020423150457.A28052@eecs.tufts.edu> <20020423155921.A142@eecs.tufts.edu> <031b01c1eb03$e012eb10$ced241d5@hagrid> <20020423205711.GA11691@panix.com> <001201c1eb0d$4adbde30$ced241d5@hagrid> <20020423213943.GA18774@panix.com> Message-ID: <200204232145.g3NLj4r14090@odiug.zope.com> > That's what I said earlier. But where is this documented? I'm not sure that every possible handy coding tip deserves to be in the documentation. There ought to be (and are!) plenty of other introductory texts that can serve this purpose. Also, in this particular case, the question ("how do I share data between modules") and the answer ("put it in a module and import it") seems pretty obvious once you have a decent grip of module semantics. --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@pythoncraft.com Tue Apr 23 22:55:03 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 23 Apr 2002 17:55:03 -0400 Subject: [Python-Dev] Re: LOAD_NAME & classes In-Reply-To: <200204232145.g3NLj4r14090@odiug.zope.com> References: <20020423130514.E10411@eecs.tufts.edu> <20020423181314.GA6816@panix.com> <20020423150457.A28052@eecs.tufts.edu> <20020423155921.A142@eecs.tufts.edu> <031b01c1eb03$e012eb10$ced241d5@hagrid> <20020423205711.GA11691@panix.com> <001201c1eb0d$4adbde30$ced241d5@hagrid> <20020423213943.GA18774@panix.com> <200204232145.g3NLj4r14090@odiug.zope.com> Message-ID: <20020423215503.GA22164@panix.com> On Tue, Apr 23, 2002, Guido van Rossum wrote: >Aahz: >> >> That's what I said earlier. But where is this documented? > > Also, in this particular case, the question ("how do I share data > between modules") and the answer ("put it in a module and import it") > seems pretty obvious once you have a decent grip of module semantics. Maybe. I personally agree. But I've seen too many people on c.l.py who appear to be definitely past the newbie level miss the boat on this topic. We'll see whether the FAQ entry helps; it should be sufficient. I noted in the FAQ entry that importing a module is also the Pythonic answer to the Singleton design pattern, so people searching for Singleton should find that. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ What if there were no rhetorical questions? From pobrien@orbtech.com Tue Apr 23 23:04:27 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Tue, 23 Apr 2002 17:04:27 -0500 Subject: [Python-Dev] Re: LOAD_NAME & classes In-Reply-To: <20020423215503.GA22164@panix.com> Message-ID: [Aahz] > Maybe. I personally agree. But I've seen too many people on c.l.py who > appear to be definitely past the newbie level miss the boat on this > topic. We'll see whether the FAQ entry helps; it should be sufficient. > I noted in the FAQ entry that importing a module is also the Pythonic > answer to the Singleton design pattern, so people searching for > Singleton should find that. Send them to Alex's article: http://www.aleax.it/5ep.html --- Patrick K. O'Brien Orbtech From aleax@aleax.it Tue Apr 23 23:09:58 2002 From: aleax@aleax.it (Alex Martelli) Date: Wed, 24 Apr 2002 00:09:58 +0200 Subject: [Python-Dev] Re: LOAD_NAME & classes In-Reply-To: References: Message-ID: <02042400095803.12462@arthur> On Wednesday 24 April 2002 00:04, Patrick K. O'Brien wrote: > [Aahz] > > > I noted in the FAQ entry that importing a module is also the Pythonic > > answer to the Singleton design pattern, so people searching for > > Singleton should find that. > > Send them to Alex's article: http://www.aleax.it/5ep.html Using a module is a better answer over 90% of the time. Most people think of Singleton "by reflex" when they believe "there's gonna be only one of these", rather than when they really need to solve the forces covered in the Gof4's Singleton DP, first and foremost subclassability. Alex From guido@python.org Wed Apr 24 01:25:14 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 23 Apr 2002 20:25:14 -0400 Subject: [Python-Dev] PEP 279 revisited Message-ID: <200204240025.g3O0PF521756@pcp742651pcs.reston01.va.comcast.net> See http://python.org/peps/pep-0279.html There's one nagging issue that I'd like to revisit. I think I still like the name itemize() better than enumerate(), even though itemize() reminds some people of items(). Somehow that doesn't strike me as a strong argument: the words look pretty different to me. I like itemize() better because (a) it's shorter, and (b) the LaTeX command "\itemize" can produce numbered lists (it can also produce bulleted lists). I'm ready to open this one for a vote. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From greg@cosc.canterbury.ac.nz Wed Apr 24 03:13:42 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Wed, 24 Apr 2002 14:13:42 +1200 (NZST) Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <200204231416.g3NEGb615386@odiug.zope.com> Message-ID: <200204240213.OAA01618@s454.cosc.canterbury.ac.nz> Guido: > def f(): > print x # user thinks this should print the global > # 2000 lines of unrelated code > for x in "some sequence": # doesn't realize this overrides x > do_something_with(x) Perhaps a warning could be issued if the first reference to a local textually precedes the first assignment to it? Would that help catch things like this, without complaining about too much existing code? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From neal@metaslash.com Wed Apr 24 03:34:41 2002 From: neal@metaslash.com (Neal Norwitz) Date: Tue, 23 Apr 2002 22:34:41 -0400 Subject: [Python-Dev] LOAD_NAME & classes References: <200204240213.OAA01618@s454.cosc.canterbury.ac.nz> Message-ID: <3CC619C1.A8661A8D@metaslash.com> Greg Ewing wrote: > > Guido: > > > def f(): > > print x # user thinks this should print the global > > # 2000 lines of unrelated code > > for x in "some sequence": # doesn't realize this overrides x > > do_something_with(x) > > Perhaps a warning could be issued if the first reference > to a local textually precedes the first assignment to it? > Would that help catch things like this, without complaining > about too much existing code? The standard library has 4 such occurrences: ftplib.py:678: Variable (macro_lines) used before being set ftplib.py:681: Variable (macro_name) used before being set tokenize.py:148: Variable (strstart) used before being set tokenize.py:149: Variable (endprog) used before being set The line numbers are from CVS version subtract 77/20 for 2.2.1. Neal From mgilfix@eecs.tufts.edu Wed Apr 24 03:37:23 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Tue, 23 Apr 2002 22:37:23 -0400 Subject: [Python-Dev] Re: LOAD_NAME & classes In-Reply-To: <200204232145.g3NLj4r14090@odiug.zope.com>; from guido@python.org on Tue, Apr 23, 2002 at 05:45:04PM -0400 References: <20020423130514.E10411@eecs.tufts.edu> <20020423181314.GA6816@panix.com> <20020423150457.A28052@eecs.tufts.edu> <20020423155921.A142@eecs.tufts.edu> <031b01c1eb03$e012eb10$ced241d5@hagrid> <20020423205711.GA11691@panix.com> <001201c1eb0d$4adbde30$ced241d5@hagrid> <20020423213943.GA18774@panix.com> <200204232145.g3NLj4r14090@odiug.zope.com> Message-ID: <20020423223723.A19604@eecs.tufts.edu> On Tue, Apr 23 @ 17:45, Guido van Rossum wrote: > I'm not sure that every possible handy coding tip deserves to be in > the documentation. There ought to be (and are!) plenty of other > introductory texts that can serve this purpose. > > Also, in this particular case, the question ("how do I share data > between modules") and the answer ("put it in a module and import it") > seems pretty obvious once you have a decent grip of module semantics. I agree now as well. I wouldn't have when I started though. At the time I wondered if I had committed a stylistic sin because it didn't feel "right". -- Mike -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From greg@cosc.canterbury.ac.nz Wed Apr 24 03:40:12 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Wed, 24 Apr 2002 14:40:12 +1200 (NZST) Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <20020423130514.E10411@eecs.tufts.edu> Message-ID: <200204240240.OAA01628@s454.cosc.canterbury.ac.nz> Michael Gilfix : > The our is akin to declaring something static in C. Except that it sounds like if two functions declare "our" variables with the same name, they get the same variable, whereas in C they would be different variables. (I think -- it just occurred to me that I'm not really sure about that!) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim.one@comcast.net Wed Apr 24 04:53:59 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 23 Apr 2002 23:53:59 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <200204231533.g3NFXcs30485@odiug.zope.com> Message-ID: [Guido] > ... > This particular form of breakage was a common error reported on c.l.py > and to help at python.org until we added UnboundLocalError to make the > diagnostic cleaner. It was indeed, and adding UnboundLocalError did cut the number of questions. > Maybe that's all that's needed; It's hard to know what could really help more. If Greg Wilson is still running newcomer experiments, I'd like to see what newcomers have to say about this: x = 2 def f(): print x # A x = 3 f() print x # B A: What do you think should happen when the print at A executes? B: " " " " " " " " " " B " ? What I suspect, but don't know, is that a majority of newcomers who expect A to print 2 *also* expect B to print 3. That is, that they're thinking x is a global variable, and have no conception of local variables in mind. This is actually what happens in Icon, which also lacks declarations (in the same sense Python lacks them: it doesn't lack them ). The difference is that all variables are global by default in Icon, and you need to explicitly say "local x" if you want a local var instead. That's error-prone for sure, by not quite as much so as Perl (where x is also global by default, but "local $x" sucks you into dynamic scoping; it does not in Icon). From ikjos@email.uophx.edu Wed Apr 24 05:01:19 2002 From: ikjos@email.uophx.edu (Ian Kjos) Date: Tue, 23 Apr 2002 23:01:19 -0500 Subject: [Python-Dev] Threading, scalability, and possible bug report Message-ID: <01c601c1eb44$b0e0a3a0$0dce4542@austin.rr.com> This is a multi-part message in MIME format. ------=_NextPart_000_01C3_01C1EB1A.C7738BC0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi. My name is Ian Kjos. I am a professional software developer by = trade, but much of the challenge has gone out of it. Some time ago, I = decided to learn the Python language. Now I believe I have something to = contribute. Barry@zope.com and I have had a short discussion, and it seems a good = time to raise the following issue on python-dev: I've been working on multithreading in python, and I have some proposals = that I would like to talk over with some knowledgable people. I aim to = donate some code to the standard library, but only if it meets the = community standards and is consistent with the goals of Python. Please read this: http://www.saltillotilestore.com/morethreading.py It contains smoke-tested code, documentation, design notes, and some = food for thought. I will not claim perfect style at the moment, but I = volunteer to clean up those issues as needed. For the impatient: Lock pools Read/Write Locks Easy Thread-Local storage support. I'm also thinking about building a metaclass for monitors (in the = data-processing sense of the word). An alternative might be a = synchronized method builder a'la classmethod and staticmethod. Lastly, before this goes standard I'll make it safe for "from X import = *". I wanted to get some feedback and direction from other developers before = continuing too far. For reference, I have already checked python = resources, and this effort seems orthogonal to other projects I could = find. /me ponders making the profiler work with threads... ------=_NextPart_000_01C3_01C1EB1A.C7738BC0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hi. My name is Ian Kjos. I am a = professional=20 software developer by trade, but much of the challenge has gone out of = it. Some=20 time ago, I decided to learn the Python language. Now I believe I have = something=20 to contribute.
 
Barry@zope.com=20 and I have had a short discussion, and it seems a good time to raise the = following issue on python-dev:
 
I've been working on multithreading in = python, and=20 I have some proposals that I would like to talk over with some = knowledgable=20 people. I aim to donate some code to the standard library, but only if = it meets=20 the community standards and is consistent with the goals of = Python.
 
Please read this:
 
http://www.sal= tillotilestore.com/morethreading.py
 
It contains smoke-tested code,=20 documentation, design notes, and some food for thought. I will = not=20 claim perfect style at the moment, but I volunteer to clean up those=20 issues as needed.
 
For the impatient:
Lock pools
Read/Write Locks
Easy Thread-Local storage = support.
 
I'm also thinking about building a = metaclass for=20 monitors (in the data-processing sense of the word). An alternative = might be a=20 synchronized method builder a'la classmethod and = staticmethod.
 
Lastly, before this goes standard I'll = make it safe=20 for "from X import *".
 
I wanted to get some feedback and = direction=20 from other developers before continuing too far. For = reference, I=20 have already checked python resources, and this effort seems = orthogonal to=20 other projects I could find.
 
/me ponders making the profiler work = with=20 threads...
 
------=_NextPart_000_01C3_01C1EB1A.C7738BC0-- From python@rcn.com Wed Apr 24 05:15:08 2002 From: python@rcn.com (Raymond Hettinger) Date: Wed, 24 Apr 2002 00:15:08 -0400 Subject: [Python-Dev] Fw: PEP 279 revisited Message-ID: <002b01c1eb46$9f588060$0cd8accf@othello> From: "Guido van Rossum" > See http://python.org/peps/pep-0279.html > > There's one nagging issue that I'd like to revisit. I think I still > like the name itemize() better than enumerate(), even though itemize() > reminds some people of items(). Somehow that doesn't strike me as a > strong argument: the words look pretty different to me. I like > itemize() better because (a) it's shorter, and (b) the LaTeX command > "\itemize" can produce numbered lists (it can also produce bulleted > lists). I'm ready to open this one for a vote. :-) Voter Information (recap of earlier discussions): iterindexed()-- five syllables is a mouthful index() -- nice verb but could be confused the .index() method indexed() -- widely liked however adjectives should be avoided indexer() -- noun did not read well in a for-loop count() -- direct and explicit but often used in other contexts itercount() -- direct, explicit and hated by more than one person iteritems() -- conflicts with key:value concept for dictionaries itemize() -- confusing because amap.items() != list(itemize(amap)) enumerate() -- does what is says. four syllables is a little long enum() -- pithy; less clear than enumerate; too similar to enum in other languages where it has a different meaning All of the names involving 'count' had the further disadvantage of implying that the count would begin from one instead of zero. All of the names involving 'index' clashed with usage in database languages where indexing implies a sorting operation rather than linear sequencing. All of the names involving 'items' clash with (key,value) concept used for dictionaries. American Heritage definition of itemize: 1. To set down item by item; list. 2. To list deductions from taxable income on a tax return. American Heritage definition of enumerate: 1. To count off or name one by one; list. 2. To determine the number of; count. Raymond Hettinger From guido@python.org Wed Apr 24 05:28:43 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 24 Apr 2002 00:28:43 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: Your message of "Wed, 24 Apr 2002 14:13:42 +1200." <200204240213.OAA01618@s454.cosc.canterbury.ac.nz> References: <200204240213.OAA01618@s454.cosc.canterbury.ac.nz> Message-ID: <200204240428.g3O4Shl22201@pcp742651pcs.reston01.va.comcast.net> > > def f(): > > print x # user thinks this should print the global > > # 2000 lines of unrelated code > > for x in "some sequence": # doesn't realize this overrides x > > do_something_with(x) > > Perhaps a warning could be issued if the first reference > to a local textually precedes the first assignment to it? > Would that help catch things like this, without complaining > about too much existing code? Absolutely, and I believe that's what PyChecker is doing. Getting it added to Python's own compiler is tricky though -- maybe we should put off all such enhancements until Jeremy's new AST-based compiler is finished. --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@pythoncraft.com Wed Apr 24 05:35:00 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 24 Apr 2002 00:35:00 -0400 Subject: [Python-Dev] Threading, scalability, and possible bug report In-Reply-To: <01c601c1eb44$b0e0a3a0$0dce4542@austin.rr.com> References: <01c601c1eb44$b0e0a3a0$0dce4542@austin.rr.com> Message-ID: <20020424043500.GC23061@panix.com> On Tue, Apr 23, 2002, Ian Kjos wrote: > > Lastly, before this goes standard I'll make it safe for "from X import *". -1 even without looking at the rest. import * is evil, evil, evil; it's a recipe for namespace pollution; it makes debugging a total chore. Okay, it's not quite that bad, but I do everything I can to discourage its use. I'll look at the rest later. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ What if there were no rhetorical questions? From guido@python.org Wed Apr 24 05:49:07 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 24 Apr 2002 00:49:07 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: Your message of "Tue, 23 Apr 2002 23:53:59 EDT." References: Message-ID: <200204240449.g3O4n7V22258@pcp742651pcs.reston01.va.comcast.net> [Tim] > It's hard to know what could really help more. If Greg Wilson is > still running newcomer experiments, I'd like to see what newcomers > have to say about this: > > > x = 2 > def f(): > print x # A > x = 3 > > f() > print x # B > > > A: What do you think should happen when the print at A executes? > B: " " " " " " " " " " B " ? > > What I suspect, but don't know, is that a majority of newcomers who > expect A to print 2 *also* expect B to print 3. That is, that > they're thinking x is a global variable, and have no conception of > local variables in mind. That's not a fair experiment until after you've given them a good concept of local and global variables without name conflicts. On the one hand, the importance of having local variables at all isn't clear if there are no name conflicts (until you introduce recursion, which I would *definitely* introduce much later, despite Matthias Felleisen's teachings :-). But on the other hand, it's easy to show that after >>> def f(): ... greeting = "hello world" ... print greeting ... >>> f() hello world >>> print greeting NameError: greeting >>> there's no global variable 'greeting', which can then be used to explain that variable assignments create local variables. You don't need to explain the *reason* for this feature at this point; that'll be clear by the time you've explained the rest. Next you can explain the global statement. This appears irrelevant to the proposed experiment, but it's useful to take away fears that you can't change globals at all: people coming from other languages will know that that is important, and worry how Python deals with this. After that you can show how name conflicts are handled in the "normal" case, where you shadow a global in a function by assigning to it and then using it, without a global statement. At some point you should also point out that if you don't *change* a global, you can use in a function it without the global statement. This comes so natural that it's easy to gloss over, especially when the "global" in question is a function or an imported module; but it *is* an important feature. *THEN* you are ready for the experiment Tim proposes above. --Guido van Rossum (home page: http://www.python.org/~guido/) From python@rcn.com Wed Apr 24 05:47:25 2002 From: python@rcn.com (Raymond Hettinger) Date: Wed, 24 Apr 2002 00:47:25 -0400 Subject: [Python-Dev] Re: PEP 279 revisited References: <200204240025.g3O0PF521756@pcp742651pcs.reston01.va.comcast.net> Message-ID: <009b01c1eb4b$218f5280$0cd8accf@othello> From: "Guido van Rossum" > There's one nagging issue that I'd like to revisit. I think I still > like the name itemize() better than enumerate(), > I'm ready to open this one for a vote. :-) One other thought. As a design principle (from The Design of Everyday Things), knowledge of what something does and how to use it should be embedded in the object not in the user. In this case, a person can learn to use either itemize() or enumerate() and bring that knowledge with them when they program. However, if the knowledge is not there (newbies), rusty (occasional users), or ambiguated (users of multiple languages), then the question arises of whether the name tells us what it does. For this reason, I vote for any name which suggests that numbers are going to come out. An arbitrary enumerate() function is clearly going to create numbers while an itemize() found in the wild could potentially do something non-numeric. So while I could personally get used to either name and be happy, I think enumerate's two extra letters or one extra syllable is worth it. 'nuff said, Raymond Hettinger From guido@python.org Wed Apr 24 05:51:08 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 24 Apr 2002 00:51:08 -0400 Subject: [Python-Dev] Threading, scalability, and possible bug report In-Reply-To: Your message of "Tue, 23 Apr 2002 23:01:19 CDT." <01c601c1eb44$b0e0a3a0$0dce4542@austin.rr.com> References: <01c601c1eb44$b0e0a3a0$0dce4542@austin.rr.com> Message-ID: <200204240451.g3O4p8C22297@pcp742651pcs.reston01.va.comcast.net> > http://www.saltillotilestore.com/morethreading.py Cool stuff, Ian! I'm no MT expert, but I hope we'll be able to do something useful with this for the benefit of those who are. --Guido van Rossum (home page: http://www.python.org/~guido/) From greg@cosc.canterbury.ac.nz Wed Apr 24 06:02:27 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Wed, 24 Apr 2002 17:02:27 +1200 (NZST) Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: <009b01c1eb4b$218f5280$0cd8accf@othello> Message-ID: <200204240502.RAA01641@s454.cosc.canterbury.ac.nz> Raymond Hettinger : > An arbitrary enumerate() function is clearly going to create > numbers But that's *not* clear to anyone who knows what the word "enumerate" really means, and isn't just guessing based on the fact that it has "num" in it. There doesn't seem to be any single English word that captures all of what we mean without ambiguity. Maybe we should be making up a new word altogether, such as "numberize" or something. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From guido@python.org Wed Apr 24 06:08:43 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 24 Apr 2002 01:08:43 -0400 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: Your message of "Wed, 24 Apr 2002 00:47:25 EDT." <009b01c1eb4b$218f5280$0cd8accf@othello> References: <200204240025.g3O0PF521756@pcp742651pcs.reston01.va.comcast.net> <009b01c1eb4b$218f5280$0cd8accf@othello> Message-ID: <200204240508.g3O58hE22360@pcp742651pcs.reston01.va.comcast.net> > One other thought. As a design principle (from The Design of > Everyday Things), knowledge of what something does and how > to use it should be embedded in the object not in the user. I don't think this applies here -- all we've got is a *name*, not an inspectable object. > In this case, a person can learn to use either itemize() or > enumerate() and bring that knowledge with them when they > program. However, if the knowledge is not there (newbies), > rusty (occasional users), or ambiguated (users of multiple > languages), then the question arises of whether the name tells > us what it does. For this reason, I vote for any name which > suggests that numbers are going to come out. Experimentation would also go a long way; even I often fire up an interactive interpreter to find out what a particular combination of operators or statements does. > An arbitrary enumerate() function is clearly going to create > numbers while an itemize() found in the wild could potentially > do something non-numeric. So while I could personally get > used to either name and be happy, I think enumerate's > two extra letters or one extra syllable is worth it. I don't know about the assertion that enumerate() is clearly going to create numbers; the word is uncommon enough that most people might not have any preconceived notion of it (certainly not people whose first language is not English), and for people coming from C, C++ or Pascal it may have a strong association with enum types, which are not about numbers (in fact they are about abstraction away from numbers ;-). Your "voter information" didn't mention this as a downside to enumerate(), but I think it's at least as important as the only argument against itemize (that it's not the same as items()). I just reviewed your implementation and noted that you abbreviated enumerate to enum everywhere, thereby closing the door for the "obvious" name (at the C API level) for an enum type, should we ever decide to add one. --Guido van Rossum (home page: http://www.python.org/~guido/) From martin@v.loewis.de Wed Apr 24 07:21:36 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 24 Apr 2002 08:21:36 +0200 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: <009b01c1eb4b$218f5280$0cd8accf@othello> References: <200204240025.g3O0PF521756@pcp742651pcs.reston01.va.comcast.net> <009b01c1eb4b$218f5280$0cd8accf@othello> Message-ID: "Raymond Hettinger" writes: > An arbitrary enumerate() function is clearly going to create=20 > numbers while an itemize() found in the wild could potentially=20 > do something non-numeric. With the highschool English education I got, I would not associate "enumerate" with "numbers are coming out". Even if it was "aufz=E4hlen", I'd still not associate it with "Zahlen" in everyday's life. Regards, Martin From aleax@aleax.it Wed Apr 24 07:46:19 2002 From: aleax@aleax.it (Alex Martelli) Date: Wed, 24 Apr 2002 08:46:19 +0200 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: <200204240508.g3O58hE22360@pcp742651pcs.reston01.va.comcast.net> References: <200204240025.g3O0PF521756@pcp742651pcs.reston01.va.comcast.net> <009b01c1eb4b$218f5280$0cd8accf@othello> <200204240508.g3O58hE22360@pcp742651pcs.reston01.va.comcast.net> Message-ID: <02042408461905.12462@arthur> On Wednesday 24 April 2002 07:08, Guido van Rossum wrote: ... > I don't know about the assertion that enumerate() is clearly going to > create numbers; the word is uncommon enough that most people might not > have any preconceived notion of it (certainly not people whose first > language is not English), It has a Latin root ("numerus" == "number"), so people whose first language is a Romance one are well placed in this way. For example, "enumerare" is a common Italian verb (with roughly the same connotations as English "enumerate", but perhaps more easily felt connection with "numero"). > and for people coming from C, C++ or Pascal > it may have a strong association with enum types, which are not about > numbers (in fact they are about abstraction away from numbers ;-). Right -- "enumerate" as in "exhaustively list" something. When one does that in the real world one often attaches numbers to items too (perhaps silently, on one's fingers) as a help in ensuring exhaustivity, but Pascal most definitely doesn't (C/C++ don't hide the numbers so much). > Your "voter information" didn't mention this as a downside to > enumerate(), but I think it's at least as important as the only > argument against itemize (that it's not the same as items()). I just I see the itemize/items issue as a _plus_ for itemize -- like items, it gives a sequence of pairs (x,v), and when the argument S is a sequence, v==S[x] -- reasonably similar (not "the same", sure, but if it WAS exactly "the same" we wouldn't have written it) to D.items() for a dict D returning a sequence [list rather than iterator] of pairs (x,v) such that v==D[x]. If the function's name didn't _have_ to be a verb (if it could follow the tradition of other builtin function names such as dir, vars, len, max and so on), naming the function "items" would emphasize this analogy. My problem with "itemize" is that in LaTex and Texinfo it means a _bulleted_ (UN-numbered) list of items. In both cases, one uses "enumerate" instead, if one wants numbers rather than bullets. But it may well be that *Tex* are too obscure to worry about that. > reviewed your implementation and noted that you abbreviated enumerate > to enum everywhere, thereby closing the door for the "obvious" name > (at the C API level) for an enum type, should we ever decide to add Sorry, my fault, not Raymond's -- I was working, as you had suggested, off the sources of iterobject.c (the 2.2.1 version, with old-fashioned GC, which doesn't excuse but may explain why I had left the GC in such a mess), and let the latter's widespread use of the abbreviations "iter_*" for "iterator" influence me into choosing "enum_*" analogously for "enumerate". Alex From aleax@aleax.it Wed Apr 24 08:07:53 2002 From: aleax@aleax.it (Alex Martelli) Date: Wed, 24 Apr 2002 09:07:53 +0200 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: <200204240502.RAA01641@s454.cosc.canterbury.ac.nz> References: <200204240502.RAA01641@s454.cosc.canterbury.ac.nz> Message-ID: <02042409075308.12462@arthur> On Wednesday 24 April 2002 07:02, Greg Ewing wrote: ... > There doesn't seem to be any single English word that > captures all of what we mean without ambiguity. Not a single VERB -- adjective "numbered" might do. Unfortunately the verb "to number" (from which the adjective comes as the past participle) is ambiguous when used as a function name (as noun "number" confusingly comes to mind instead, since it's an usage vastly more common). Alex From xscottg@yahoo.com Wed Apr 24 08:37:03 2002 From: xscottg@yahoo.com (Scott Gilbert) Date: Wed, 24 Apr 2002 00:37:03 -0700 (PDT) Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: Message-ID: <20020424073703.10786.qmail@web12907.mail.yahoo.com> --- Tim Peters wrote: > > This is actually what happens in Icon, which also lacks declarations (in > the same sense Python lacks them: it doesn't lack them ). The > difference > is that all variables are global by default in Icon, and you need to > explicitly say "local x" if you want a local var instead. That's > error-prone for sure, by not quite as much so as Perl (where x is also > global by default, but "local $x" sucks you into dynamic scoping; it does > not in Icon). > A few corrections. New variable names were local by default. The only way to get a global variable was to make a "global varname" statement at the global scope. Icon gave you a None-like value, &null, for your variables if you hadn't already assigned to them somewhere. So strictly speaking, you could read a variable before writing to it, and a declaration (first initialization) wasn't necessary the same way it is in Python. Using "local x" was strongly recommended because (a) you could use the -u switch of icont to warn you when you didn't declare your locals (usually from typos), and (b) in case you forgot that you used the same name as a global somewhere else, or more likely you "link"ed with a file that had global declarations you weren't aware of. So it really wasn't so error prone. (Icon's not even close to Perl in this regard. :-) Cheers, -Scott __________________________________________________ Do You Yahoo!? Yahoo! Games - play chess, backgammon, pool and more http://games.yahoo.com/ From aleax@aleax.it Wed Apr 24 08:38:39 2002 From: aleax@aleax.it (Alex Martelli) Date: Wed, 24 Apr 2002 09:38:39 +0200 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <200204240240.OAA01628@s454.cosc.canterbury.ac.nz> References: <200204240240.OAA01628@s454.cosc.canterbury.ac.nz> Message-ID: <0204240938390B.12462@arthur> On Wednesday 24 April 2002 04:40, Greg Ewing wrote: > Michael Gilfix : > > The our is akin to declaring something static in C. > > Except that it sounds like if two functions declare > "our" variables with the same name, they get the > same variable, whereas in C they would be different > variables. (I think -- it just occurred to me that > I'm not really sure about that!) You think correctly: two C functions declaring static function-scope variables with the same name get different variables. static variables at FILE scope are 'shared' throughout the file, but not between files. Alex From xscottg@yahoo.com Wed Apr 24 08:49:08 2002 From: xscottg@yahoo.com (Scott Gilbert) Date: Wed, 24 Apr 2002 00:49:08 -0700 (PDT) Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: <02042409075308.12462@arthur> Message-ID: <20020424074908.96023.qmail@web12901.mail.yahoo.com> --- Alex Martelli wrote: > ... > > There doesn't seem to be any single English word that > > captures all of what we mean without ambiguity. > > Not a single VERB -- adjective "numbered" might do. Unfortunately > [...] If there is no really good name, how about just adopting a new simple, short, word like "each", and letting it become the idiom? for i, v in each(L): print i, v It worked for "zip" right? __________________________________________________ Do You Yahoo!? Yahoo! Games - play chess, backgammon, pool and more http://games.yahoo.com/ From aleax@aleax.it Wed Apr 24 10:19:18 2002 From: aleax@aleax.it (Alex Martelli) Date: Wed, 24 Apr 2002 11:19:18 +0200 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: <20020424074908.96023.qmail@web12901.mail.yahoo.com> References: <20020424074908.96023.qmail@web12901.mail.yahoo.com> Message-ID: On Wednesday 24 April 2002 09:49 am, Scott Gilbert wrote: ... > > > There doesn't seem to be any single English word that > > > captures all of what we mean without ambiguity. > > > > Not a single VERB -- adjective "numbered" might do. Unfortunately > > [...] > > If there is no really good name, how about just adopting a new simple, > short, word like "each", and letting it become the idiom? > > for i, v in each(L): > print i, v > > It worked for "zip" right? I'd just love something like this, but unfortunately the precedent you cite probably does not apply: 'zip' is indeed a verb, 'each' is, alas, not. What's now named 'enumerate', incidentally, is a type (just like 'tuple', 'dict', 'str' -- all consistently non-verbs, and types rather than functions since a few months), so I'm not quite sure why it should be a verb (what other types' names are verbs, except for the slight ambiguity of 'list' or 'file', which I've always perceived in the noun-sense anyway?). Of course, if a noun was to be chosen instead, 'enumeration' and 'itemization' would be horrid (long and cumbersome), but maybe some other alternatives could be more appealing. 'items', for example, would be appealing to me personally (by the already mentioned "rough analogy" with a dictionary's .items method -- which turns others off by being only rough and not exact, so that D.items() and items(D) would be different sequences of pairs, albeit with similar structures, for just about any dict D). Alex From aahz@pythoncraft.com Wed Apr 24 12:37:23 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 24 Apr 2002 07:37:23 -0400 Subject: FWD: Re: [Python-Dev] Fw: PEP 279 revisited Message-ID: <20020424113723.GA6556@panix.com> Raymond says Guido is handling the vote. ----- Forwarded message from Aahz ----- > Date: Wed, 24 Apr 2002 00:30:28 -0400 > From: Aahz > To: Raymond Hettinger > Subject: Re: [Python-Dev] Fw: PEP 279 revisited > >> enumerate() -- does what is says. four syllables is a little long > > +1 ----- End forwarded message ----- -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ What if there were no rhetorical questions? From pyth@devel.trillke.net Wed Apr 24 12:35:10 2002 From: pyth@devel.trillke.net (holger krekel) Date: Wed, 24 Apr 2002 13:35:10 +0200 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: References: <20020424074908.96023.qmail@web12901.mail.yahoo.com> Message-ID: <20020424113510.GN27395@devel.trillke> On Wed, Apr 24, 2002 at 11:19:18AM +0200, Alex Martelli wrote: > Of course, if a noun was to be chosen instead, 'enumeration' and > 'itemization' would be horrid (long and cumbersome), but maybe > some other alternatives could be more appealing. 'items', for example, > would be appealing to me personally (by the already mentioned > "rough analogy" with a dictionary's .items method -- which turns > others off by being only rough and not exact, so that D.items() > and items(D) would be different sequences of pairs, albeit with > similar structures, for just about any dict D). I would like to transform the 'rough' analogy into an exact analogy :-) If you consider lists to be a special dictionary with implicit keys we can define 'items' to generate (key,value) pairs for a collection so that for key,value in items(collection): collection[key]==value holds no matter if collection is a list or a dictionary, Additionally items(dict)==dict.items() would be satisfied, so the analogy is better than 'rough'. btw, i usually don't need an extra numbering for my dictionaries. If really in need i would write list = dict.items() for index,item in items(list): ... using the above semantics. Usually i use a list in the first place. But i am afraid i am too late if this is the final voting... holger -- P.S: As i am new to the list, i introduce myself here. Mailman told me to :-) I am using python only for a year or so but it is clearly the most productive language i have ever encountered. I am extremely happy to be able participate and contribute in c.l.py and the dev-list. Code and oppinions! A very big "thank you" to everyone here!! Coming from Assembler,c++,java -worlds i am doing lots of stuff with distributed systems. I am about to release an open source CORBA Transaction Service in C++ (http://xots.org). Sometimes i write for german computer magazines CT/iX. In the latter there will be an article about EuroPython :-) So much for now, feel free to ask, also in personal mail ... Holger From barry@zope.com Wed Apr 24 12:46:32 2002 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 24 Apr 2002 07:46:32 -0400 Subject: FWD: Re: [Python-Dev] Fw: PEP 279 revisited References: <20020424113723.GA6556@panix.com> Message-ID: <15558.39704.517471.946745@anthem.wooz.org> I guess since we're voting, I still like enumerate() best. Reason: it smells nice. Justification of reason: I think we've about plumbed all the reason(s) out of this decision, and all we're left with is personal preference, and I like names that smell nice. -Barry From barry@zope.com Wed Apr 24 13:00:51 2002 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 24 Apr 2002 08:00:51 -0400 Subject: [Python-Dev] Re: PEP 279 revisited References: <20020424074908.96023.qmail@web12901.mail.yahoo.com> <20020424113510.GN27395@devel.trillke> Message-ID: <15558.40563.661863.722171@anthem.wooz.org> >>>>> "hk" == holger krekel writes: hk> P.S: As i am new to the list, i introduce myself here. hk> Mailman told me to :-) Holger, I predict you'll work out great here. You're already following the first rule, which as the Timbot will tell you, is to always slavish do whatever Mailman tells you to do. It works for me, and not just because whenever I deviate from its plans, it smacks me on the head and forces me back into the party line. -Barry From Damien.Morton@acm.org Wed Apr 24 13:04:06 2002 From: Damien.Morton@acm.org (Damien Morton) Date: Wed, 24 Apr 2002 08:04:06 -0400 Subject: [Python-Dev] re: PEP 279 revisited Message-ID: <00b301c1eb88$225a56f0$56296c42@damien> Theres one word that was left out from the discussion. numerate() As far as I can tell, the dictionary meaning is pretty much the same as for enumerate, but theres a much stronger association with numbering things (especially for non-latin speakers). From aleax@aleax.it Wed Apr 24 13:10:47 2002 From: aleax@aleax.it (Alex Martelli) Date: Wed, 24 Apr 2002 14:10:47 +0200 Subject: [Python-Dev] re: PEP 279 revisited In-Reply-To: <00b301c1eb88$225a56f0$56296c42@damien> References: <00b301c1eb88$225a56f0$56296c42@damien> Message-ID: <02042414104702.13425@arthur> On Wednesday 24 April 2002 14:04, Damien Morton wrote: > Theres one word that was left out from the discussion. > > numerate() > > As far as I can tell, the dictionary meaning is pretty much the same as > for enumerate, but theres a much stronger association with numbering > things (especially for non-latin speakers). I think the main use of 'numerate' is as an adjective meaning roughly the same as 'literate' but about numbers/maths rather than about 'letters' (words/writing/reading/...). Didn't know it was a verb in English (in Italian the equivalent "numerare" exists, and means "associate numbers with" more strictly than "enumerare" does, so it _would_ be perfect for a native speaker of Italian -- hardly an important issue, though). Alex From mwh@python.net Wed Apr 24 12:13:12 2002 From: mwh@python.net (Michael Hudson) Date: 24 Apr 2002 12:13:12 +0100 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: Tim Peters's message of "Tue, 23 Apr 2002 12:34:48 -0400" References: Message-ID: <2m662hl587.fsf@starship.python.net> Tim Peters writes: > [Michael Hudson, on UnboundLocalErrors] > > ... > > Wouldn't it be nice if these were flagged at compile time! More work > > for Jeremy . > > As will (or already has ) been pointed out, the compiler can only > guess, and can't always guess right (it could err on the side of bothering > you without cause, though). That's what I was implying, yes. > PyChecker goes a long way already. Java has a mass of "definite > assignment" rules instead, which in effect decree that you must > write code in such a way that the compiler can always guess right. Are these explained in an easy to digest form somewhere? Cheers, M. -- I saw `cout' being shifted "Hello world" times to the left and stopped right there. -- Steve Gonedes From aleax@aleax.it Wed Apr 24 13:24:20 2002 From: aleax@aleax.it (Alex Martelli) Date: Wed, 24 Apr 2002 14:24:20 +0200 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <2m662hl587.fsf@starship.python.net> References: <2m662hl587.fsf@starship.python.net> Message-ID: <02042414242004.13425@arthur> On Wednesday 24 April 2002 13:13, Michael Hudson wrote: ... > > PyChecker goes a long way already. Java has a mass of "definite > > assignment" rules instead, which in effect decree that you must > > write code in such a way that the compiler can always guess right. > > Are these explained in an easy to digest form somewhere? http://java.sun.com/docs/books/jls/second_edition/html/defAssign.doc.html Alex From Damien.Morton@acm.org Wed Apr 24 13:27:32 2002 From: Damien.Morton@acm.org (Damien Morton) Date: Wed, 24 Apr 2002 08:27:32 -0400 Subject: [Python-Dev] re: PEP 279 revisited In-Reply-To: <02042414104702.13425@arthur> Message-ID: <00b601c1eb8b$687f28b0$56296c42@damien> http://www.dictionary.com/search?q=numerate ------------------------ tr.v. nu.mer.at.ed, nu.mer.at.ing, nu.mer.ates To enumerate; count. adj. (-mr-t) Able to think and express oneself effectively in quantitative terms. Source: The American HeritageR Dictionary of the English Language, Fourth Edition ------------------------- \Nu"mer*ate\, v. t. [imp. & p. p. Numerated; p. pr. & vb. n. Numerating.] [L. numeratus, p. p. of numerare to count. See Number, v.] (Arith.) To divide off and read according to the rules of numeration; as, to numerate a row of figures. Source: Webster's Revised Unabridged Dictionary, C 1996, 1998 MICRA, Inc. ------------- Of course, the fundamental operation here is numbering things in a list. list.number() perhaps? Unfortunatly, the noun/verb ambiguity exists with that suggestion, but... I like numerare, it would be a fine suggestion if more of the world spoke Italian. > -----Original Message----- > From: Alex Martelli [mailto:aleax@aleax.it] > Sent: Wednesday, 24 April 2002 08:11 > To: Damien Morton; python-dev@python.org > Subject: Re: [Python-Dev] re: PEP 279 revisited > > > On Wednesday 24 April 2002 14:04, Damien Morton wrote: > > Theres one word that was left out from the discussion. > > > > numerate() > > > > As far as I can tell, the dictionary meaning is pretty much > the same > > as for enumerate, but theres a much stronger association with > > numbering things (especially for non-latin speakers). > > I think the main use of 'numerate' is as an adjective meaning > roughly the same as 'literate' but about numbers/maths rather > than about 'letters' (words/writing/reading/...). Didn't > know it was a verb in English (in Italian the equivalent > "numerare" exists, and means "associate numbers > with" more strictly than "enumerare" does, so it _would_ be > perfect for a native speaker of Italian -- hardly an > important issue, though). > > > Alex > From neal@metaslash.com Wed Apr 24 13:26:03 2002 From: neal@metaslash.com (Neal Norwitz) Date: Wed, 24 Apr 2002 08:26:03 -0400 Subject: [Python-Dev] Re: PEP 279 revisited References: <20020424074908.96023.qmail@web12901.mail.yahoo.com> <20020424113510.GN27395@devel.trillke> Message-ID: <3CC6A45B.C7480568@metaslash.com> holger krekel wrote: [discussion on naming enumerate() items()] > Additionally > > items(dict)==dict.items() > > would be satisfied, so the analogy is better than 'rough'. They are not equivalent, see below. Neal -- >>> d = {10:20} >>> print d.items() [(10, 20)] >>> print enumerate(d) >>> for x in enumerate(d): print x ... (0, 10) From barry@zope.com Wed Apr 24 13:31:17 2002 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 24 Apr 2002 08:31:17 -0400 Subject: [Python-Dev] Threading, scalability, and possible bug report References: <01c601c1eb44$b0e0a3a0$0dce4542@austin.rr.com> <20020424043500.GC23061@panix.com> Message-ID: <15558.42389.543358.874479@anthem.wooz.org> >>>>> "A" == Aahz writes: A> import * is evil, evil, evil; it's a recipe for namespace A> pollution; it makes debugging a total chore. But like bare-excepts, it has it's place. -Barry From guido@python.org Wed Apr 24 13:37:22 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 24 Apr 2002 08:37:22 -0400 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: Your message of "Wed, 24 Apr 2002 00:49:08 PDT." <20020424074908.96023.qmail@web12901.mail.yahoo.com> References: <20020424074908.96023.qmail@web12901.mail.yahoo.com> Message-ID: <200204241237.g3OCbMD23048@pcp742651pcs.reston01.va.comcast.net> > If there is no really good name, how about just adopting a new simple, > short, word like "each", and letting it become the idiom? > > for i, v in each(L): > print i, v -1. each() doesn't convey any helpful connotations. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Apr 24 13:47:25 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 24 Apr 2002 08:47:25 -0400 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: Your message of "Wed, 24 Apr 2002 13:35:10 +0200." <20020424113510.GN27395@devel.trillke> References: <20020424074908.96023.qmail@web12901.mail.yahoo.com> <20020424113510.GN27395@devel.trillke> Message-ID: <200204241247.g3OClPV23143@pcp742651pcs.reston01.va.comcast.net> > If you consider lists to be a special dictionary > with implicit keys we can define 'items' to > generate (key,value) pairs for a collection so that > > for key,value in items(collection): > collection[key]==value > > holds no matter if collection is a list or a dictionary, This has been proposed and rejected before. We've made the decision that "iter(dict)" iterates over the keys while "iter(list)" iterates over the values, and anything that tries to unify the two types (really categories of types: sequences and mappings) is bound to fail. > btw, i usually don't need an extra numbering for my dictionaries. > If really in need i would write > > list = dict.items() > for index,item in items(list): ... > > using the above semantics. That's another argument why what this function (whatever it's called) does on dicts is irrelevant: you wouldn't want to use it on a dict anyway. It's something for sequences and iterators. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Apr 24 13:49:27 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 24 Apr 2002 08:49:27 -0400 Subject: [Python-Dev] re: PEP 279 revisited In-Reply-To: Your message of "Wed, 24 Apr 2002 08:04:06 EDT." <00b301c1eb88$225a56f0$56296c42@damien> References: <00b301c1eb88$225a56f0$56296c42@damien> Message-ID: <200204241249.g3OCnRn23155@pcp742651pcs.reston01.va.comcast.net> > Theres one word that was left out from the discussion. > > numerate() > > As far as I can tell, the dictionary meaning is pretty much the same as > for enumerate, but theres a much stronger association with numbering > things (especially for non-latin speakers). I'm +/- 0 on this. It saves a letter and may indeed suggest numbering more closely, but it's an even more obscure word than enumerate(). BTW, the two votes I've received so far were in favor of enumerate(). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Apr 24 13:57:14 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 24 Apr 2002 08:57:14 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: Your message of "24 Apr 2002 12:13:12 BST." <2m662hl587.fsf@starship.python.net> References: <2m662hl587.fsf@starship.python.net> Message-ID: <200204241257.g3OCvEK23177@pcp742651pcs.reston01.va.comcast.net> > > PyChecker goes a long way already. Java has a mass of "definite > > assignment" rules instead, which in effect decree that you must > > write code in such a way that the compiler can always guess right. > > Are these explained in an easy to digest form somewhere? Dunno, but I can easily guess the gist of it: a variable is only defined on paths if it would still always be defined if all conditional statements randomly chose between true or false, and all loops executed a random number of times which may include zero. I assume that this means dependencies between variables are out (these made the examples reported by Neil work). --Guido van Rossum (home page: http://www.python.org/~guido/) From jacobs@penguin.theopalgroup.com Wed Apr 24 14:11:37 2002 From: jacobs@penguin.theopalgroup.com (Kevin Jacobs) Date: Wed, 24 Apr 2002 09:11:37 -0400 (EDT) Subject: [Python-Dev] Threading, scalability, and possible bug report In-Reply-To: <01c601c1eb44$b0e0a3a0$0dce4542@austin.rr.com> Message-ID: On Tue, 23 Apr 2002, Ian Kjos wrote: > I'm also thinking about building a metaclass for monitors (in the > data-processing sense of the word). An alternative might be a synchronized > method builder a'la classmethod and staticmethod. I already have a metaclass for monitors in the works. I'm about to go out of town for a week, but I'll definitly send you my current prototype when I return. -Kevin -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com From nas@python.ca Wed Apr 24 14:37:29 2002 From: nas@python.ca (Neil Schemenauer) Date: Wed, 24 Apr 2002 06:37:29 -0700 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: <200204240502.RAA01641@s454.cosc.canterbury.ac.nz>; from greg@cosc.canterbury.ac.nz on Wed, Apr 24, 2002 at 05:02:27PM +1200 References: <009b01c1eb4b$218f5280$0cd8accf@othello> <200204240502.RAA01641@s454.cosc.canterbury.ac.nz> Message-ID: <20020424063728.A29666@glacier.arctrix.com> Greg Ewing wrote: > There doesn't seem to be any single English word that > captures all of what we mean without ambiguity. How about "indices"? You use a key to get things out of dictionaries. You use an index to get things out of sequences. "indices" is the pural of index. Neil From Anthony Baxter Wed Apr 24 14:59:38 2002 From: Anthony Baxter (Anthony Baxter) Date: Wed, 24 Apr 2002 23:59:38 +1000 Subject: FWD: Re: [Python-Dev] Fw: PEP 279 revisited In-Reply-To: Message from barry@zope.com (Barry A. Warsaw) of "Wed, 24 Apr 2002 07:46:32 -0400." <15558.39704.517471.946745@anthem.wooz.org> Message-ID: <200204241359.g3ODxcR07695@localhost.localdomain> fwiw, I prefer enumerate() over itemize(). There's no real reason to pick one over the other - I just think "enumerate()" is more different, and therefore clearer (to me). I had hoped Mr Shorter Oxford would help here, but the definitions are both applicable. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From Anthony Baxter Wed Apr 24 15:01:05 2002 From: Anthony Baxter (Anthony Baxter) Date: Thu, 25 Apr 2002 00:01:05 +1000 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: Message from barry@zope.com (Barry A. Warsaw) of "Wed, 24 Apr 2002 08:00:51 -0400." <15558.40563.661863.722171@anthem.wooz.org> Message-ID: <200204241401.g3OE15907724@localhost.localdomain> >>> Barry A. Warsaw wrote > always slavish do whatever Mailman tells you to do. It works for me, > and not just because whenever I deviate from its plans, it smacks me > on the head and forces me back into the party line. Great. So Barry does what Mailman tells him to do, and Barry modifies and improves Mailman. I predict a self-aware Mailman that kills us all within 2 years. From pyth@devel.trillke.net Wed Apr 24 15:03:41 2002 From: pyth@devel.trillke.net (holger krekel) Date: Wed, 24 Apr 2002 16:03:41 +0200 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: <200204241247.g3OClPV23143@pcp742651pcs.reston01.va.comcast.net> References: <20020424074908.96023.qmail@web12901.mail.yahoo.com> <20020424113510.GN27395@devel.trillke> <200204241247.g3OClPV23143@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020424140341.GO27395@devel.trillke> On Wed, Apr 24, 2002 at 08:47:25AM -0400, Guido van Rossum wrote: > > If you consider lists to be a special dictionary > > with implicit keys we can define 'items' to > > generate (key,value) pairs for a collection so that > > > > for key,value in items(collection): > > collection[key]==value > > > > holds no matter if collection is a list or a dictionary, > > This has been proposed and rejected before. We've made the decision > that "iter(dict)" iterates over the keys while "iter(list)" iterates > over the values, and anything that tries to unify the two types > (really categories of types: sequences and mappings) is bound to fail. ok, i am fine with that. after all i can always define my own 'items' or maybe better 'keyvalues' method to achieve the unification where needed :-) regarding PEP279 i go with 'itemize' ... holger From aleax@aleax.it Wed Apr 24 15:25:39 2002 From: aleax@aleax.it (Alex Martelli) Date: Wed, 24 Apr 2002 16:25:39 +0200 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: <20020424063728.A29666@glacier.arctrix.com> References: <009b01c1eb4b$218f5280$0cd8accf@othello> <200204240502.RAA01641@s454.cosc.canterbury.ac.nz> <20020424063728.A29666@glacier.arctrix.com> Message-ID: On Wednesday 24 April 2002 03:37 pm, Neil Schemenauer wrote: > Greg Ewing wrote: > > There doesn't seem to be any single English word that > > captures all of what we mean without ambiguity. > > How about "indices"? You use a key to get things out of dictionaries. > You use an index to get things out of sequences. "indices" is the pural > of index. Yes, but it's a noun, not a verb. Again, I don't understand why this type's name should be a verb, but apparently that's the Decision -- we're only being consulted on "which verb". Apart from this, "indices" suggests you're getting ONLY indices -- while when you iterate on this type you get indices AND contents. In other words, name "indices" might be fine for a hypothetical different type, for usage such as: for i in indices(mysequence): x = mysequence[i] # etc rather than the current: for i in xrange(len(mysequence)): x = mysequence[i] # etc but the type RH and I implemented (not without help from GvR in fixing the mess I'd made of GC &c:-) is to be used differently: for i, x in mysterynamegoeshere(mysequence_or_other_iterator): # just the etc -- x, the i-th value, is already in hand Alex From mgilfix@eecs.tufts.edu Wed Apr 24 15:29:49 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Wed, 24 Apr 2002 10:29:49 -0400 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: <20020424063728.A29666@glacier.arctrix.com>; from nas@python.ca on Wed, Apr 24, 2002 at 06:37:29AM -0700 References: <009b01c1eb4b$218f5280$0cd8accf@othello> <200204240502.RAA01641@s454.cosc.canterbury.ac.nz> <20020424063728.A29666@glacier.arctrix.com> Message-ID: <20020424102947.B19604@eecs.tufts.edu> I'm glad I came across this post because I was about to write it myself. indices () seems like a good explanation of what's going on. It returns both the index and what it corresponds to. Much better than enumerate. -- Mike On Wed, Apr 24 @ 06:37, Neil Schemenauer wrote: > Greg Ewing wrote: > > There doesn't seem to be any single English word that > > captures all of what we mean without ambiguity. > > How about "indices"? You use a key to get things out of dictionaries. > You use an index to get things out of sequences. "indices" is the pural > of index. > > Neil > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev `-> (nas) -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From niemeyer@conectiva.com Wed Apr 24 15:52:34 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Wed, 24 Apr 2002 11:52:34 -0300 Subject: [Python-Dev] PEP 279 revisited In-Reply-To: <200204240025.g3O0PF521756@pcp742651pcs.reston01.va.comcast.net> References: <200204240025.g3O0PF521756@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020424115234.A1493@ibook.distro.conectiva> > There's one nagging issue that I'd like to revisit. I think I still > like the name itemize() better than enumerate(), even though itemize() > reminds some people of items(). Somehow that doesn't strike me as a > strong argument: the words look pretty different to me. I like > itemize() better because (a) it's shorter, and (b) the LaTeX command > "\itemize" can produce numbered lists (it can also produce bulleted > lists). I'm ready to open this one for a vote. :-) +1 on enumerate(). I must confess I didn't understand the idea just looking at that function's name. But once I've read the intention, I found the function name perfect for the job. Btw, your second argument seems to strike against you. AFAIK, \begin{itemize} on LaTeX creates bulleted lists. To create enumerated lists you must use \begin{enumerate}. ;-) -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From pyth@devel.trillke.net Wed Apr 24 16:04:18 2002 From: pyth@devel.trillke.net (holger krekel) Date: Wed, 24 Apr 2002 17:04:18 +0200 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: <200204241247.g3OClPV23143@pcp742651pcs.reston01.va.comcast.net> References: <20020424074908.96023.qmail@web12901.mail.yahoo.com> <20020424113510.GN27395@devel.trillke> <200204241247.g3OClPV23143@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020424150418.GP27395@devel.trillke> On Wed, Apr 24, 2002 at 08:47:25AM -0400, Guido van Rossum wrote: > > btw, i usually don't need an extra numbering for my dictionaries. > > If really in need i would write > > > > list = dict.items() > > for index,item in items(list): ... > > > > using the above semantics. > > That's another argument why what this function (whatever it's called) > does on dicts is irrelevant: you wouldn't want to use it on a dict > anyway. It's something for sequences and iterators. Is it really a bad idea then to make 'this function' return something useful for mapping types? This could also resolve the 'only rough analogy' problem for the name 'itemize' versus 'items'. (Although these two are really different words it is a little confusing even to demi-gods:-) holger From guido@python.org Wed Apr 24 16:12:25 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 24 Apr 2002 11:12:25 -0400 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: Your message of "Wed, 24 Apr 2002 17:04:18 +0200." <20020424150418.GP27395@devel.trillke> References: <20020424074908.96023.qmail@web12901.mail.yahoo.com> <20020424113510.GN27395@devel.trillke> <200204241247.g3OClPV23143@pcp742651pcs.reston01.va.comcast.net> <20020424150418.GP27395@devel.trillke> Message-ID: <200204241512.g3OFCPn23665@pcp742651pcs.reston01.va.comcast.net> > Is it really a bad idea then to make 'this function' return > something useful for mapping types? Yes. Believe me. Don't go there. (Sigh. I was only asking for guidance on whether to choose enumerate or itemize for the name, not reopening the whole subject of this PEP.) --Guido van Rossum (home page: http://www.python.org/~guido/) From bfordham@socialistsushi.com Wed Apr 24 16:27:54 2002 From: bfordham@socialistsushi.com (Bryan L. Fordham) Date: Wed, 24 Apr 2002 11:27:54 -0400 (EDT) Subject: [Python-Dev] Re: PEP 279 revisited Message-ID: <20020424112708.U73550-100000@the.whole.net> Guido van Rossum wrote: > (Sigh. I was only asking for guidance on whether to choose enumerate > or itemize for the name, not reopening the whole subject of this PEP.) Let the language decide: >>> choice(['enumerate', 'itemize']) 'enumerate' 8) --B From pobrien@orbtech.com Wed Apr 24 16:34:23 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Wed, 24 Apr 2002 10:34:23 -0500 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: Message-ID: [Alex Martelli] > > What's now named 'enumerate', incidentally, is a type (just like 'tuple', > 'dict', 'str' -- all consistently non-verbs, and types rather > than functions > since a few months), so I'm not quite sure why it should be a verb I'm partial to enumerate, but agree with Alex that since it returns a type the name is less than satisfying. So I'd like to float the idea of using the Iterator spec as our model for this new type and function. The new type is a numerator but that is a mouthful so the built-in function should be named numer(), just like iter() is short for iterator. If people can learn and remember iter(), it should be just as easy to learn and remember numer(). They are spelled almost the same, they rhyme, they are short, etc. All the logic that lead to the selection of iter() should apply here. --- Patrick K. O'Brien Orbtech From barry@zope.com Wed Apr 24 16:35:51 2002 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 24 Apr 2002 11:35:51 -0400 Subject: [Python-Dev] Re: PEP 279 revisited References: <15558.40563.661863.722171@anthem.wooz.org> <200204241401.g3OE15907724@localhost.localdomain> Message-ID: <15558.53463.846756.393269@anthem.wooz.org> >>>>> "AB" == Anthony Baxter writes: AB> Great. So Barry does what Mailman tells him to do, and Barry AB> modifies and improves Mailman. I predict a self-aware Mailman AB> that kills us all within 2 years. You're thinking of the older Postman protocol (specifically GPP - the Go Postal Protocol), which was indeed prone to fits of violence. The newer MHHJJP (Mailman Happy Happy Joy Joy Protocol) wouldn't kill anybody or anything. It might turn you all into 'bots, but as the Tim and /F bots have shown, that may not be such a bad thing. -B From magnus@hetland.org Wed Apr 24 16:39:56 2002 From: magnus@hetland.org (Magnus Lie Hetland) Date: Wed, 24 Apr 2002 17:39:56 +0200 Subject: [Python-Dev] PEP 279 revisited In-Reply-To: <200204240025.g3O0PF521756@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Tue, Apr 23, 2002 at 08:25:14PM -0400 References: <200204240025.g3O0PF521756@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020424173956.I28260@idi.ntnu.no> Guido van Rossum : > > See http://python.org/peps/pep-0279.html > > There's one nagging issue that I'd like to revisit. I think I still > like the name itemize() better than enumerate(), even though itemize() > reminds some people of items(). Somehow that doesn't strike me as a > strong argument: the words look pretty different to me. I like > itemize() better because (a) it's shorter, and (b) the LaTeX command > "\itemize" can produce numbered lists (it can also produce bulleted > lists). I'm ready to open this one for a vote. :-) Well, the itemize environment can create any kind of list, but you'll have to number the items manually... ;) How about enum()? That's short, and sort of in the traditoin of iter()... Can either be interpreted as a noun, an adjective, or a verb... > --Guido van Rossum (home page: http://www.python.org/~guido/) -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.org From aleax@aleax.it Wed Apr 24 16:41:03 2002 From: aleax@aleax.it (Alex Martelli) Date: Wed, 24 Apr 2002 17:41:03 +0200 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: References: Message-ID: On Wednesday 24 April 2002 05:34 pm, Patrick K. O'Brien wrote: ... > I'm partial to enumerate, but agree with Alex that since it returns a type It IS a type; calling it returns an instance of the type. (Calling any callable returns an instance of SOME type, after all:-). > The new type is a numerator but that is a mouthful so the built-in function > should be named numer(), just like iter() is short for iterator. If people Nice. I doubt it will fly, but among the alternatives mentioned so far it gets my vote. Alex From magnus@hetland.org Wed Apr 24 16:43:22 2002 From: magnus@hetland.org (Magnus Lie Hetland) Date: Wed, 24 Apr 2002 17:43:22 +0200 Subject: [Python-Dev] PEP 279 revisited In-Reply-To: <20020424173956.I28260@idi.ntnu.no>; from magnus@hetland.org on Wed, Apr 24, 2002 at 05:39:56PM +0200 References: <200204240025.g3O0PF521756@pcp742651pcs.reston01.va.comcast.net> <20020424173956.I28260@idi.ntnu.no> Message-ID: <20020424174322.A18146@idi.ntnu.no> Magnus Lie Hetland : [snip] > How about enum()? That's short, and sort of in the traditoin of > iter()... Can either be interpreted as a noun, an adjective, or a > verb... Oops... Should have read the list of existing suggestions more thoroughly. Sorry. (Anyway... If possible, I vote for enum() -- otherwise, I vote for itemize(), exactly because of the "analogy" to dict.items) -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.org From fredrik@pythonware.com Wed Apr 24 15:02:27 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 24 Apr 2002 16:02:27 +0200 Subject: [Python-Dev] re: PEP 279 revisited References: <00b601c1eb8b$687f28b0$56296c42@damien> Message-ID: <028901c1eba6$ebab11c0$0900a8c0@spiff> Damien wrote: > Of course, the fundamental operation here is numbering things in a = list. no, it's not: it's about numbering things in a sequence. there is no sequence base class, and thus no place to put that number method. From fredrik@pythonware.com Wed Apr 24 16:36:34 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 24 Apr 2002 17:36:34 +0200 Subject: [Python-Dev] Threading, scalability, and possible bug report References: <01c601c1eb44$b0e0a3a0$0dce4542@austin.rr.com><20020424043500.GC23061@panix.com> <15558.42389.543358.874479@anthem.wooz.org> Message-ID: <028a01c1eba6$ebcca380$0900a8c0@spiff> barry wrote: > >>>>> "A" =3D=3D Aahz writes: >=20 > A> import * is evil, evil, evil; it's a recipe for namespace > A> pollution; it makes debugging a total chore. >=20 > But like bare-excepts, it has it's place. the rule is "use only if you know what you're doing", not "don't use because aahz don't know what you're doing" From paul@svensson.org Wed Apr 24 16:57:12 2002 From: paul@svensson.org (Paul Svensson) Date: Wed, 24 Apr 2002 11:57:12 -0400 (EDT) Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: <200204241512.g3OFCPn23665@pcp742651pcs.reston01.va.comcast.net> Message-ID: On Wed, 24 Apr 2002, Guido van Rossum wrote: >> Is it really a bad idea then to make 'this function' return >> something useful for mapping types? > >Yes. Believe me. Don't go there. So should it explicitly raise a TypeError then, to avoid accidentally returning something (less than) useful ? my +1 for calling it itemize, btw /Paul From David Abrahams" Message-ID: <12b101c1ebab$36edbe90$6501a8c0@boostconsulting.com> ----- Original Message ----- From: "Alex Martelli" > > The new type is a numerator but that is a mouthful so the built-in function > > should be named numer(), just like iter() is short for iterator. If people > > Nice. I doubt it will fly, but among the alternatives mentioned so far it > gets my vote. Blick. Abbrevs suck. Plus I don't want to re-use the sucky name enum for something related to what 'C' and C++ use it for. +1 for itemize +0.8 for enumerate (in other contexts/languages/systems it just returns the items in sequence like a python iterator does). OK, so round it up to 1 if fractions are illegal. -Dave From aahz@pythoncraft.com Wed Apr 24 17:23:39 2002 From: aahz@pythoncraft.com (Aahz) Date: Wed, 24 Apr 2002 12:23:39 -0400 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: References: Message-ID: <20020424162339.GA14974@panix.com> On Wed, Apr 24, 2002, Patrick K. O'Brien wrote: > > The new type is a numerator but that is a mouthful so the built-in > function should be named numer(), just like iter() is short for > iterator. If people can learn and remember iter(), it should be just > as easy to learn and remember numer(). They are spelled almost the > same, they rhyme, they are short, etc. All the logic that lead to the > selection of iter() should apply here. Hrm. I'm not quite changing my vote yet, but numerator() might work; I don't like numer() because it looks like it's missing a "b". I'm also rethinking whether I should cast my vote for iterindexed(). -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ What if there were no rhetorical questions? From paul@prescod.net Wed Apr 24 17:38:53 2002 From: paul@prescod.net (Paul Prescod) Date: Wed, 24 Apr 2002 09:38:53 -0700 Subject: [Python-Dev] re: PEP 279 revisited References: <00b301c1eb88$225a56f0$56296c42@damien> <200204241249.g3OCnRn23155@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CC6DF9D.38254BF9@prescod.net> Guido van Rossum wrote: > >... > > I'm +/- 0 on this. It saves a letter and may indeed suggest numbering > more closely, but it's an even more obscure word than enumerate(). Is not obscurity a sort of virtue? If it hasn't been used in the languages space then it has no mental baggage. Compare with "Big endian", as obscure a reference as you'll find in computer science, I think. Or "thunk": a made-up word. Paul Prescod From xscottg@yahoo.com Wed Apr 24 17:41:52 2002 From: xscottg@yahoo.com (Scott Gilbert) Date: Wed, 24 Apr 2002 09:41:52 -0700 (PDT) Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: <200204241512.g3OFCPn23665@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020424164152.23581.qmail@web12905.mail.yahoo.com> --- Guido van Rossum wrote: > > (Sigh. I was only asking for guidance on whether to choose enumerate > or itemize for the name, not reopening the whole subject of this PEP.) > /Smile. When I read the first message in this thread, I strongly suspected you threw this out there to verify the whole "the smaller the issue, the more people who will comment" theory that came up in the discussion of bools. I still strongly suspect that. :-) __________________________________________________ Do You Yahoo!? Yahoo! Games - play chess, backgammon, pool and more http://games.yahoo.com/ From DavidA@ActiveState.com Wed Apr 24 18:05:15 2002 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 24 Apr 2002 10:05:15 -0700 Subject: [Python-Dev] Re: PEP 279 revisited References: <200204240025.g3O0PF521756@pcp742651pcs.reston01.va.comcast.net> <009b01c1eb4b$218f5280$0cd8accf@othello> <200204240508.g3O58hE22360@pcp742651pcs.reston01.va.comcast.net> <02042408461905.12462@arthur> Message-ID: <3CC6E5CB.9020006@ActiveState.com> Alex Martelli wrote: >On Wednesday 24 April 2002 07:08, Guido van Rossum wrote: > ... > >>I don't know about the assertion that enumerate() is clearly going to >>create numbers; the word is uncommon enough that most people might not >>have any preconceived notion of it (certainly not people whose first >>language is not English), >> > >It has a Latin root ("numerus" == "number"), so people whose first language >is a Romance one are well placed in this way. > Which is probably why I brought up this point originally. I'll be damned. My latin teacher was right. Latin and etymology classes _were_ going to be useful after all! =) --davidatum ascherus From nas@python.ca Wed Apr 24 18:20:38 2002 From: nas@python.ca (Neil Schemenauer) Date: Wed, 24 Apr 2002 10:20:38 -0700 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: ; from aleax@aleax.it on Wed, Apr 24, 2002 at 04:25:39PM +0200 References: <009b01c1eb4b$218f5280$0cd8accf@othello> <200204240502.RAA01641@s454.cosc.canterbury.ac.nz> <20020424063728.A29666@glacier.arctrix.com> Message-ID: <20020424102038.A30471@glacier.arctrix.com> Alex Martelli wrote: > Apart from this, "indices" suggests you're getting ONLY indices -- > while when you iterate on this type you get indices AND contents. Ah, I misunderstood what this unnamed function does. I was thinking it works like .keys() on dictionaries. You're telling me it works like .items(). In that case I don't like enumerate() or itemize(). :-( Neil From DavidA@ActiveState.com Wed Apr 24 18:37:57 2002 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 24 Apr 2002 10:37:57 -0700 Subject: [Python-Dev] re: PEP 279 revisited, formally References: <00b301c1eb88$225a56f0$56296c42@damien> <200204241249.g3OCnRn23155@pcp742651pcs.reston01.va.comcast.net> <3CC6DF9D.38254BF9@prescod.net> Message-ID: <3CC6ED75.7030208@ActiveState.com> Paul Prescod wrote: >Guido van Rossum wrote: > >>... >> >>I'm +/- 0 on this. It saves a letter and may indeed suggest numbering >>more closely, but it's an even more obscure word than enumerate(). >> > >Is not obscurity a sort of virtue? If it hasn't been used in the >languages space then it has no mental baggage. Compare with "Big >endian", as obscure a reference as you'll find in computer science, I >think. Or "thunk": a made-up word. > Let's try and approach this from a formal POV. For a representative population of target users, if they see code using name X, what is the likelihood of (a) - guessing the meaning correctly ; (b) - guessing the meaning incorrectly; (c) - refusing to guess and looking it up. Now design a cost function C(a,b,c|X), and minimize over X. The details are in the cost function particulars: Paul's cost function says that what matters is minimizing b, even if it means increasing c. A made-up word or one with no 'likely to guess' meaning, is safest as far as that cost function is concerned. Admittedly, it doesn't take into account the cost of looking things up. Another cost function would be to maximize (a-b), ignoring c. That is what people seem to be 'voting' on, I think, although more analysis would be needed to make that a strong claim. There seem to be anecdotal data flying (again =) indicating that if either enumerate or itemize are chosen for X, a and b are likely to be similar in magnitude and would dwarf c. Given that, the two cost functions yield very different data. What cost function do we want to apply to such things? Do we have previous experiences with bad names that had bad consequences? My personal "naming" problems in Python has to do with never remembering the order of the two arguments to pickle.dump(). I get it wrong about 50% of the time if I haven't been pickling recently. Along the same lines, I have no problem remembering the name of the pickle module, even though its relationship to the semantics are fairly abstract at best. I'm reluctant to vote between enumerate and itemize because I think Paul's onto something. I'll close with an anecdote which is even harder to apply to the case at hand. My mom recently got her first Windows PC after using macs for years. I'm talking to her on the phone: "How do I throw things off of my desktop?" "Just drag them to the trash can" "But there's no trash can ... . oh, there's a recycle bin -- but I do't want to recycle, I want to throw it away!" --da From paul@prescod.net Wed Apr 24 18:55:22 2002 From: paul@prescod.net (Paul Prescod) Date: Wed, 24 Apr 2002 10:55:22 -0700 Subject: [Python-Dev] re: PEP 279 revisited, formally References: <00b301c1eb88$225a56f0$56296c42@damien> <200204241249.g3OCnRn23155@pcp742651pcs.reston01.va.comcast.net> <3CC6DF9D.38254BF9@prescod.net> <3CC6ED75.7030208@ActiveState.com> Message-ID: <3CC6F18A.5AAB5384@prescod.net> David Ascher wrote: > >... > Let's try and approach this from a formal POV. My only addition to your formal model is a factor for mnemonic-ness. I like "numerate" because I wouldn't have a bunch of preconditions but once I figured out what it meant I'd remember it. Paul Prescod From sholden@holdenweb.com Wed Apr 24 18:56:28 2002 From: sholden@holdenweb.com (Steve Holden) Date: Wed, 24 Apr 2002 13:56:28 -0400 Subject: [Python-Dev] re: PEP 279 revisited, formally References: <00b301c1eb88$225a56f0$56296c42@damien> <200204241249.g3OCnRn23155@pcp742651pcs.reston01.va.comcast.net> <3CC6DF9D.38254BF9@prescod.net> <3CC6ED75.7030208@ActiveState.com> Message-ID: <03da01c1ebb9$5c6dc760$d200000a@holdenweb.com> David Ascher wrote: > > I'll close with an anecdote which is even harder to apply to the case at > hand. My mom recently got her first Windows PC after using macs for > years. I'm talking to her on the phone: > > "How do I throw things off of my desktop?" > "Just drag them to the trash can" > "But there's no trash can ... . oh, there's a recycle bin -- but I > do't want to recycle, I want to throw it away!" > Given that Mac users eject removable media by dragging the media's icon onto the trashcan, presumably there is already some ambiguity in their minds about what the trashcan does. An interesting conversation often results from asking Americans what the difference is between trash and garbage. Given that the answer is "garbage contains food waste", perhaps we should really trash-collect objects when their refcount becomes zero? I now return you to this important semantic discussion <0.85 wink>. unless-of-course-you-eat-objects-ly y'rs - steve -- home: http://www.holdenweb.com/ Python Web Programming: http://pydish.holdenweb.com/pwp/ From just@letterror.com Wed Apr 24 19:53:06 2002 From: just@letterror.com (Just van Rossum) Date: Wed, 24 Apr 2002 20:53:06 +0200 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: <20020424102038.A30471@glacier.arctrix.com> Message-ID: Alex Martelli wrote: > Apart from this, "indices" suggests you're getting ONLY indices -- > while when you iterate on this type you get indices AND contents. Neil Schemenauer wrote: > Ah, I misunderstood what this unnamed function does. I was thinking it > works like .keys() on dictionaries. You're telling me it works like > ..items(). In that case I don't like enumerate() or itemize(). :-( Eh, this is the definition from the PEP: def enumerate(collection): 'Generates an indexed series: (0,coll[0]), (1,coll[1]) ...' i = 0 it = iter(collection) while 1: yield (i, it.next()) i += 1 (And enumerate() gets a +1 from me.) Just From mclay@nist.gov Wed Apr 24 21:18:26 2002 From: mclay@nist.gov (Michael McLay) Date: Wed, 24 Apr 2002 16:18:26 -0400 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: <20020424112708.U73550-100000@the.whole.net> References: <20020424112708.U73550-100000@the.whole.net> Message-ID: <200204242018.g3OKIGvU004922@email.nist.gov> On Wednesday 24 April 2002 11:27 am, Bryan L. Fordham wrote: > Guido van Rossum wrote: > > (Sigh. I was only asking for guidance on whether to choose enumerate > > or itemize for the name, not reopening the whole subject of this PEP.) > > Let the language decide: > >>> choice(['enumerate', 'itemize']) > > 'enumerate' > > 8) Hmm, I used a sample size of 8 and it looks like a landslide for 'itemize'. >>> choice(['enumerate', 'itemize']) 'enumerate' >>> choice(['enumerate', 'itemize']) 'itemize' >>> choice(['enumerate', 'itemize']) 'itemize' >>> choice(['enumerate', 'itemize']) 'itemize' >>> choice(['enumerate', 'itemize']) 'itemize' >>> choice(['enumerate', 'itemize']) 'itemize' >>> choice(['enumerate', 'itemize']) 'itemize' >>> choice(['enumerate', 'itemize']) 'enumerate' None of the choices in the PEP jump out as being ideal. Perhaps itemize is a better choice simply because it is shorter. Advantages? - It takes up less space in a line of code. - It is easier to type. +0 for itemize From ikjos@email.uophx.edu Wed Apr 24 21:40:51 2002 From: ikjos@email.uophx.edu (Ian Kjos) Date: Wed, 24 Apr 2002 15:40:51 -0500 Subject: [Python-Dev] pep 279 rehashed Message-ID: <007701c1ebd0$52f35c10$cc058dd8@avatartech.com> /me attempts to avoid HTML-mail. +1 for numer() in the short run, but I don't know if I get a vote yet. I don't know if it's too late yet, but what PEP 279 really seems to want is iterator comprehensions. I have wanted those myself for a bit of a while. If nobody minds, how about these classes: Counter -> iterator over unbounded arithmetic sequence, with specified start and optional arbitrary step. IterComp -> convert a transform function, a list of iterators and an optional selection function into a new lazy iterator, using comprehension-type semantics. Expect frequent use of lambdas. Wait! Isn't this just like programming with streams? Is there room in the python model for an explicit stream type? (Yes, it's called a generator. Perhaps we should just use those wisely.) From James_Althoff@i2.com Wed Apr 24 21:49:37 2002 From: James_Althoff@i2.com (James_Althoff@i2.com) Date: Wed, 24 Apr 2002 13:49:37 -0700 Subject: [Python-Dev] re: PEP 279 revisited Message-ID: I defined both in the interactive interpreter and typed (and then looked at) each one ten times. "enumerate" was annoying to type and didn't looked that great either. "itemize" seemed fine. +1 on "itemize" Jim From tim.one@comcast.net Thu Apr 25 05:33:01 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 25 Apr 2002 00:33:01 -0400 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: Message-ID: [Martin v. Loewis] > With the highschool English education I got, I would not associate > "enumerate" with "numbers are coming out". Even if it was "aufz= =E4hlen", > I'd still not associate it with "Zahlen" in everyday's life. So +1 on zahlen() for this function's name . From tim.one@comcast.net Thu Apr 25 05:35:01 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 25 Apr 2002 00:35:01 -0400 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: <200204241401.g3OE15907724@localhost.localdomain> Message-ID: [Anthony Baxter] > I predict a self-aware Mailman that kills us all within 2 years. I'm afraid you're 3 years too late. all-your-you-are-belong-to-mailman-ly y'rs - tim From tim.one@comcast.net Thu Apr 25 06:08:49 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 25 Apr 2002 01:08:49 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <200204240449.g3O4n7V22258@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido, on > x = 2 > def f(): > print x # A > x = 3 > > f() > print x # B > > > A: What do you think should happen when the print at A executes? > B: " " " " " " " " " " B " ? ] > That's not a fair experiment until after you've given them a good > concept of local and global variables without name conflicts. I think my point is getting lost twice: 1. My suspicion is that the majority of people I've explained this to, over the years and years, would have answered the equivalent to "well, B should print 3, of course!", when translated to the specific context in which their confusion arose. That is, I suspect they didn't understand local variables at all, not they that had some vision of "well, it's the global name unless and until I dynamically override it with a local name, at which point the global name with the same spelling is no longer relevant". 2. If we're talking about people willing to be taught "a good concept of local and global variables" before going postal, there's no "problem" to be solved here! I suspect even Mr. Kuchling understands how local variables work today . > ... > But on the other hand, it's easy to show that after ... > which can then be used to explain that variable assignments create > local variables. > You don't need to explain the *reason* for this feature at this point .. > Next you can explain the global statement ... > After that you can show how name conflicts are handled ... > At some point you should also point out that ... > *THEN* you are ready for the experiment Tim proposes above. Anyone willing to sit through that much explanation will have no problem understanding how locals work today. Besides, if you get to tell them beforehand that "variable assigments *create* local variables" dynamically, then I get to tell them beforehand that variable assignments don't create local variables dynamically. At that point we'd only be testing how faithfully they can regurgitate what they've just been told. I'm not a sure a difference between 12% comprehension rate and 13% comprehension rate is statistically significant anyway <0.9 wink>. From tim.one@comcast.net Thu Apr 25 07:06:21 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 25 Apr 2002 02:06:21 -0400 Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: <20020424073703.10786.qmail@web12907.mail.yahoo.com> Message-ID: [Scott Gilbert, on Icon] > A few corrections. New variable names were local by default. Ack! You're right! Years of stuffing in "local" decls made me forget why they were recommended to begin with. > The only way to get a global variable was to make a "global varname" > statement at the global scope. The difficulty being-- why explicit "local" was really recommended --that a new global magically changes every *implicit* local vrbl of the same name into a global. This bit too when an implicit local happened to have the same name as a new function added later, because function names are automatically global. Like: procedure main() f() g() end procedure f() count := 1 while count < 10 do count +:= 1 write(count) end procedure g() write(len([2, 3, 4]) | "oops!") end procedure len(x) return *x end That prints 10 3 If you just happen to change the name of the "len" function to "count" later: procedure g() write(count([2, 3, 4]) | "oops!") end procedure count(x) return *x end it suddenly prints 10 oops! instead (the expression count([2, 3, 4]) | "oops!" becomes 10([2, 3, 4]) | "oops!" because the once-local "count" in f is suddenly global, and retains its value of 10; then 10([2, 3, 4]) fails, and the alternative "oops!" is tried and succeeds). In real life these could be an absolute bitch to track down, so I always used local. Insert local count at the top of f and everything's fine again. > Icon gave you a None-like value, &null, for your variables if you > hadn't already assigned to them somewhere. So strictly speaking, > you could read a variable before writing to it, and a declaration (first > initialization) wasn't necessary the same way it is in Python. I prefer Python distinguishing between None and unbound too; e.g., I'd much rather have x := y blow up at once if y hasn't been bound than to silently propagate &null (or a Python None). &null could travel *all over the place* before you finally hit an expression where it didn't just silently fail (very few things in Icon raise errors, as the 10([2, 3, 4]) example should have suggested to onlookers ). > Using "local x" was strongly recommended because (a) you could use the > -u switch of icont to warn you when you didn't declare your locals > (usually from typos), Unfortunately, -u doesn't complain about the "10 oops!" example above: as far as it's concerned, all variables were declared in that example. Fortunately, it does complain about the original way of writing it, because "count" is undeclared then. > and (b) in case you forgot that you used the same name as a global > somewhere else, or more likely you "link"ed with a file that had global > declarations you weren't aware of. Like an implicit "import *" in Python -- we're well aware of the dangers of *that* around here . > So it really wasn't so error prone. (Icon's not even close to > Perl in this regard. :-) Not for scope rules alone, no, although Perl sticking to explicit "my" everywhere isn't much worse than Icon sticking to explicit "local" everywere. From aleax@aleax.it Wed Apr 24 21:23:17 2002 From: aleax@aleax.it (Alex Martelli) Date: Wed, 24 Apr 2002 22:23:17 +0200 Subject: [Python-Dev] Re: PEP 279 revisited In-Reply-To: <12b101c1ebab$36edbe90$6501a8c0@boostconsulting.com> References: <12b101c1ebab$36edbe90$6501a8c0@boostconsulting.com> Message-ID: <02042422231703.13684@arthur> On Wednesday 24 April 2002 18:13, David Abrahams wrote: ... > > > should be named numer(), just like iter() is short for iterator. If ... > Blick. Abbrevs suck. Plus I don't want to re-use the sucky name enum for > something related to what 'C' and C++ use it for. This proposal was for 'numer', not 'enum'. And, given a language that has a 'def' keyword, and built-ins called 'dict', 'iter', 'vars', 'bool', 'str', 'repr', etc (:-), the affermation "abbrevs suck" looks pretty peculiar to me. Note that a few of these are pretty new, too, so it's not a question of "legacy" -- clearly the designer _prefers_ to use abbreviations in certain places he deems suitable. Alex From Paul.Moore@atosorigin.com Thu Apr 25 09:15:51 2002 From: Paul.Moore@atosorigin.com (Moore, Paul) Date: Thu, 25 Apr 2002 09:15:51 +0100 Subject: [Python-Dev] Re: PEP 279 revisited Message-ID: <714DFA46B9BBD0119CD000805FC1F53B01B5B2E4@UKRUX002.rundc.uk.origin-it.com> I'm not sure my vote counts for much (I mainly just lurk here), but I prefer enumerate(). The one thing that no-one's mentioned yet, but which will probably cause trouble, is that itemize() could be spelled itemise(), depending on where you're from. People can get used to this sort of thing (heck, I even type "color" without stopping to wonder what I did wrong these days :-) but it's an unnecessary annoyance. So I'm +1 for enumerate. I'm sorry - I can't think of a new name to suggest. Does this disqualify me from voting? Paul From xscottg@yahoo.com Thu Apr 25 09:30:27 2002 From: xscottg@yahoo.com (Scott Gilbert) Date: Thu, 25 Apr 2002 01:30:27 -0700 (PDT) Subject: [Python-Dev] LOAD_NAME & classes In-Reply-To: Message-ID: <20020425083027.61389.qmail@web12908.mail.yahoo.com> --- Tim Peters wrote: > > instead (the expression > > count([2, 3, 4]) | "oops!" > > becomes > > 10([2, 3, 4]) | "oops!" > > because the once-local "count" in f is suddenly global, and retains its > value of 10; then 10([2, 3, 4]) fails, and the alternative "oops!" is > tried > and succeeds). > Yep that's ugly. I never wrote any programs in Icon larger than a couple thousand lines, and I never used Icon in a team setting, so I didn't get bit by things like this. > > I prefer Python distinguishing between None and unbound too; e.g., I'd > much > rather have > > x := y > > blow up at once if y hasn't been bound than to silently propagate &null > I have mixed feelings on this one. I remember using the / and \ operators all over the place to work around it, so in that regard I guess I didn't like it. On the other hand it nicely parallels how tables would return &null (or other default) for keys that weren't already in there, and that I liked. In Python, I use the d.get('key', default) construct quite a bit. Probably because I learned to like getting a default value from Icon. I could use a DefaultDict class derived from dict, but that's less convenient and doesn't help me with **kwds arguments. Plus it's a hindrance to someone reading my code who knows what Pythonic behavior is supposed to be. No one says that key lookup in dicts/tables and variable lookup in the namespace have to follow the same rules though. It just seems that they do in both Icon and Python. Each with a different choice for rules. > > (very few things in > Icon raise errors, as the 10([2, 3, 4]) example should have suggested to > onlookers ). > I took two Icon courses from Ralph, and several times while sitting in class, I got the impression that he regretted a lot of things like this. Having integers usable in the place of functions was odd and rarely useful. He also didn't seem to have much respect for co-expressions. He was really excited about the graphics features though. Now *strings* as functions, that I used. :-) I think part of the charm of Icon was* that it felt like an experiment to see what cool new concepts come out of letting anything do everything. When it comes time to teach it or use it in a large project, having the language force a little more bondage is probably a good thing. * - I should say "charm of Icon is". It looks like it's still got something of a pulse in academia... In fact the latest version just built under Cygwin without major headaches. > > Unfortunately, -u doesn't complain about the "10 oops!" example above: > as > Yeah, a -U-and-I-really-mean-it flag would fix this. :-) Maybe if procedures were "const" the problem would go away. (While probably creating some new problem that I'm not thinking of...) Trying to draw this back as relevant to Python and this thread: I think Icon has an easier time with scoping rules than Python because execution starts in "procedure main()", not the beginning of the file. Since code isn't floating out there at the topmost level, one can't create variables there without an explicit global declaration. There really are only two scopes for global and local. (And some builtins of course...) Following this thread and reading some of the examples, I don't think I have a deep grasp of what the rules are in Python. I just have a set of conventions that I follow which seem to keep me out of trouble. Your example of: x = 2 def f(): print x #x = 3 ### uncomment to go BOOM kind of surprised me. I presented it to another co-worker who wasn't very Python savvy. It caught me off guard that he expected the exception at the print statement, but he didn't realize it only occurs when you have the "local declaration" later. > > Like an implicit "import *" in Python -- we're well aware of the dangers > of *that* around here . > Yep, and I litter my libraries with underscores to protect from it. A number of users at my company gripe about having to use parens and commas for function calls, especially in interactive mode, so convincing them to use module.function(arg, arg, arg) is going to be a really tough sell... (Someone suggested the LazyPython hack as a sort of solution to the first problem, but they would prefer "import module" defaulted to the bad behavior) The underscores also make it easy to see when something is global or not, avoiding the BOOM in your example above. > > So it really wasn't so error prone. (Icon's not even close to > > Perl in this regard. :-) > > Not for scope rules alone, no, although Perl sticking to explicit "my" > everywhere isn't much worse than Icon sticking to explicit "local" > everywhere. > My nostalgia for Icon makes me forget about any of the bad things. I don't have much nostalgia for Perl, so it's faults I remember. Cheers, -Scott __________________________________________________ Do You Yahoo!? Yahoo! Games - play chess, backgammon, pool and more http://games.yahoo.com/ From thomas.heller@ion-tof.com Thu Apr 25 13:46:13 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 25 Apr 2002 14:46:13 +0200 Subject: [Python-Dev] METH_CLASS Message-ID: <045901c1ec57$2f0a5240$e000a8c0@thomasnotebook> The devel-docs tell me about METH_CLASS: The method will be passed the type object as the first parameter rather than an instance of the type. This is used to create class methods, similar to what is created when using the classmethod() built-in function. New in version 2.3. 1. This seems wrong. The first parameter the function receives is NULL, the type object goes together with the remaining arguments in the second parameter (which is a tuple). Is the documentation wrong, or is the code wrong (I didn't find any use of METH_CLASS in the CVS sources)? [I must say that I'm not really sure how this relates to the calling convention flags. Have tried METH_O and METH_VARARGS so far.] 2. Since this is marked new in 2.3: Is it planned to backport this stuff into the 2.2.x series? Requiring Python 2.3 for my stuff seems a but harsh currently... 3. While we're at it, although I could try to find out myself: How would one create a classmethod in Python 2.2? Thanks, Thomas From thomas.heller@ion-tof.com Thu Apr 25 13:57:10 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 25 Apr 2002 14:57:10 +0200 Subject: [Python-Dev] METH_CLASS References: <045901c1ec57$2f0a5240$e000a8c0@thomasnotebook> Message-ID: <047601c1ec58$b669d1b0$e000a8c0@thomasnotebook> I wrote: > 3. While we're at it, although I could try to find out myself: > How would one create a classmethod [in C] in Python 2.2? and I answered it myself with a variant of my standard answer: Implement it as an instance method in the metaclass. Thomas From guido@python.org Thu Apr 25 13:59:39 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 25 Apr 2002 08:59:39 -0400 Subject: [Python-Dev] METH_CLASS In-Reply-To: Your message of "Thu, 25 Apr 2002 14:46:13 +0200." <045901c1ec57$2f0a5240$e000a8c0@thomasnotebook> References: <045901c1ec57$2f0a5240$e000a8c0@thomasnotebook> Message-ID: <200204251259.g3PCxdk25615@pcp742651pcs.reston01.va.comcast.net> > The devel-docs tell me about METH_CLASS: > > The method will be passed the type object as the first parameter > rather than an instance of the type. This is used to create class > methods, similar to what is created when using the classmethod() > built-in function. New in version 2.3. > > 1. This seems wrong. The first parameter the function receives > is NULL, the type object goes together with the remaining arguments > in the second parameter (which is a tuple). > > Is the documentation wrong, or is the code wrong (I didn't find > any use of METH_CLASS in the CVS sources)? I'll leave this for Fred to answer, since he implemented this. > [I must say that I'm not really sure how this relates to the > calling convention flags. Have tried METH_O and METH_VARARGS > so far.] > > 2. Since this is marked new in 2.3: Is it planned to backport > this stuff into the 2.2.x series? > Requiring Python 2.3 for my stuff seems a but harsh currently... I don't plan to backport this to 2.2 unless there's an overwhelming demand. > 3. While we're at it, although I could try to find out myself: > How would one create a classmethod in Python 2.2? You have to call classmethod() from C; it's available as &PyClassMethod_Type, you can call it with e.g. PyObject_Call. The argument should be a function object that you created by calling PyCFunction_New. Then stick it into your type's tp_dict (after calling PyType_Ready()). I know that's complicated; that's why we added METH_CLASS. :-) I'd recommend against trying this unless you absolutely need it. --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas.heller@ion-tof.com Thu Apr 25 14:26:11 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 25 Apr 2002 15:26:11 +0200 Subject: [Python-Dev] METH_CLASS References: <045901c1ec57$2f0a5240$e000a8c0@thomasnotebook> <200204251259.g3PCxdk25615@pcp742651pcs.reston01.va.comcast.net> Message-ID: <04a101c1ec5c$c453d7e0$e000a8c0@thomasnotebook> From: "Guido van Rossum" > > The devel-docs tell me about METH_CLASS: [...] > > 2. Since this is marked new in 2.3: Is it planned to backport > > this stuff into the 2.2.x series? > > Requiring Python 2.3 for my stuff seems a but harsh currently... > > I don't plan to backport this to 2.2 unless there's an overwhelming > demand. What do you accept as overwhelming? The only person I could think of currently beside me is David Abrahams, but maybe I'm wrong ;-) > You have to call classmethod() from C; it's available as > &PyClassMethod_Type, you can call it with e.g. PyObject_Call. The > argument should be a function object that you created by calling > PyCFunction_New. Then stick it into your type's tp_dict (after > calling PyType_Ready()). > > I know that's complicated; that's why we added METH_CLASS. :-) > I'd recommend against trying this unless you absolutely need it. I need it, so I asked :-) But I'm fine with your recipe (in these cases where I don't have a custom metaclass). Thanks, Thomas From thomas.heller@ion-tof.com Thu Apr 25 15:14:01 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 25 Apr 2002 16:14:01 +0200 Subject: [Python-Dev] METH_CLASS References: <045901c1ec57$2f0a5240$e000a8c0@thomasnotebook> <200204251259.g3PCxdk25615@pcp742651pcs.reston01.va.comcast.net> Message-ID: <057401c1ec63$72fc4f60$e000a8c0@thomasnotebook> > > The devel-docs tell me about METH_CLASS: > > > > The method will be passed the type object as the first parameter > > rather than an instance of the type. This is used to create class > > methods, similar to what is created when using the classmethod() > > built-in function. New in version 2.3. > > > > 1. This seems wrong. The first parameter the function receives > > is NULL, the type object goes together with the remaining arguments > > in the second parameter (which is a tuple). > > > > Is the documentation wrong, or is the code wrong (I didn't find > > any use of METH_CLASS in the CVS sources)? > > I'll leave this for Fred to answer, since he implemented this. If there is still a choice, I would prefer that the docs are correct, and the code should be fixed. This would allow to use, for example, METH_O in the usual way: a function which receives one parameter (in addition to the type object passed implicitely). Thomas From fdrake@acm.org Thu Apr 25 15:20:04 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 25 Apr 2002 10:20:04 -0400 Subject: [Python-Dev] METH_CLASS In-Reply-To: <045901c1ec57$2f0a5240$e000a8c0@thomasnotebook> References: <045901c1ec57$2f0a5240$e000a8c0@thomasnotebook> <200204251259.g3PCxdk25615@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15560.4244.263234.167462@grendel.zope.com> Thomas Heller writes: > Is the documentation wrong, or is the code wrong (I didn't find > any use of METH_CLASS in the CVS sources)? There's an example in sandbox/datetime/datetime.c; search for "datetime_now". > [I must say that I'm not really sure how this relates to the > calling convention flags. Have tried METH_O and METH_VARARGS > so far.] Guido van Rossum writes: > You have to call classmethod() from C; it's available as > &PyClassMethod_Type, you can call it with e.g. PyObject_Call. The > argument should be a function object that you created by calling > PyCFunction_New. Then stick it into your type's tp_dict (after > calling PyType_Ready()). I'll note that while this is a little tedious, it's easily packaged into a helper function. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From thomas.heller@ion-tof.com Thu Apr 25 15:40:53 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 25 Apr 2002 16:40:53 +0200 Subject: [Python-Dev] METH_CLASS References: <045901c1ec57$2f0a5240$e000a8c0@thomasnotebook><200204251259.g3PCxdk25615@pcp742651pcs.reston01.va.comcast.net> <15560.4244.263234.167462@grendel.zope.com> Message-ID: <05d501c1ec67$34257060$e000a8c0@thomasnotebook> From: "Fred L. Drake, Jr." > > Thomas Heller writes: > > Is the documentation wrong, or is the code wrong (I didn't find > > any use of METH_CLASS in the CVS sources)? > > There's an example in sandbox/datetime/datetime.c; search for > "datetime_now". Yes, it is also used (demonstrated) in Modules/xxsubtype.c. But this doesn't answer my question. I still think that the docs and the code disagree. (and also I _hope_ that the docs are correct and the code is wrong ;-) Thomas From fdrake@acm.org Thu Apr 25 15:55:50 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 25 Apr 2002 10:55:50 -0400 Subject: [Python-Dev] METH_CLASS In-Reply-To: <05d501c1ec67$34257060$e000a8c0@thomasnotebook> References: <045901c1ec57$2f0a5240$e000a8c0@thomasnotebook> <200204251259.g3PCxdk25615@pcp742651pcs.reston01.va.comcast.net> <15560.4244.263234.167462@grendel.zope.com> <05d501c1ec67$34257060$e000a8c0@thomasnotebook> Message-ID: <15560.6390.863304.817007@grendel.zope.com> Thomas Heller writes: > Yes, it is also used (demonstrated) in Modules/xxsubtype.c. I'd describe that as an incredibly degenerate case. ;-) > But this doesn't answer my question. I still think that > the docs and the code disagree. (and also I _hope_ that > the docs are correct and the code is wrong ;-) Guido & I just had a little pow-wow about this, and decided that this is worth changing. I'll make the changes to the code so that the documentation is right. Can you please file a bug report on this so I don't lose track of it? It should be assigned to me, priority 6. (So that I actually need to review your distutils patch first. ;) Thanks for reading the development docs and bringing this up! -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From guido@python.org Thu Apr 25 15:54:06 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 25 Apr 2002 10:54:06 -0400 Subject: [Python-Dev] METH_CLASS In-Reply-To: Your message of "Thu, 25 Apr 2002 16:40:53 +0200." <05d501c1ec67$34257060$e000a8c0@thomasnotebook> References: <045901c1ec57$2f0a5240$e000a8c0@thomasnotebook><200204251259.g3PCxdk25615@pcp742651pcs.reston01.va.comcast.net> <15560.4244.263234.167462@grendel.zope.com> <05d501c1ec67$34257060$e000a8c0@thomasnotebook> Message-ID: <200204251454.g3PEs6E15340@odiug.zope.com> > But this doesn't answer my question. I still think that > the docs and the code disagree. (and also I _hope_ that > the docs are correct and the code is wrong ;-) Fred & I had a little whiteboard session. The docs indeed disagree with the code, and we agree that the code should be fixed. This requires some work: we'll have to introduce a new kind of descriptor that wraps a PyMethodDef pointer and creates a PyCFunction in response to the tp_descr_get call with the type as self. Fred will do this. --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas.heller@ion-tof.com Thu Apr 25 16:20:47 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 25 Apr 2002 17:20:47 +0200 Subject: [Python-Dev] METH_CLASS References: <045901c1ec57$2f0a5240$e000a8c0@thomasnotebook><200204251259.g3PCxdk25615@pcp742651pcs.reston01.va.comcast.net> <15560.4244.263234.167462@grendel.zope.com> <05d501c1ec67$34257060$e000a8c0@thomasnotebook> <200204251454.g3PEs6E15340@odiug.zope.com> Message-ID: <064601c1ec6c$c689d770$e000a8c0@thomasnotebook> From: "Guido van Rossum" > > But this doesn't answer my question. I still think that > > the docs and the code disagree. (and also I _hope_ that > > the docs are correct and the code is wrong ;-) > > Fred & I had a little whiteboard session. The docs indeed disagree > with the code, and we agree that the code should be fixed. This > requires some work: we'll have to introduce a new kind of descriptor > that wraps a PyMethodDef pointer and creates a PyCFunction in response > to the tp_descr_get call with the type as self. Fred will do this. > Great. I filed a bug report http://www.python.org/sf/548651. From: "Fred L. Drake, Jr." > > Thanks for reading the development docs and bringing this up! I do this for pure egoistic reasons ;-) Thomas From neal@metaslash.com Thu Apr 25 16:42:52 2002 From: neal@metaslash.com (Neal Norwitz) Date: Thu, 25 Apr 2002 11:42:52 -0400 Subject: [Python-Dev] More deprecations Message-ID: <3CC823FC.CF687436@metaslash.com> Should the use of deprecated functions in the stdlib be removed? string.atoi -> int string.atof -> float sys.exc_trackback -> sys.exc_info()[2] Should the methods which use gopher also be deprecated? ConfigParser.py:311: string.atoi is deprecated ConfigParser.py:314: string.atof is deprecated Cookie.py:331: string.atoi is deprecated inspect.py:658: sys.exc_traceback is deprecated urllib2.py:103: Module gopherlib is deprecated urllib.py:381: Module gopherlib is deprecated (Line #s aren't exactly correct, but problems exist in CVS.) Neal From guido@python.org Thu Apr 25 16:56:16 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 25 Apr 2002 11:56:16 -0400 Subject: [Python-Dev] More deprecations In-Reply-To: Your message of "Thu, 25 Apr 2002 11:42:52 EDT." <3CC823FC.CF687436@metaslash.com> References: <3CC823FC.CF687436@metaslash.com> Message-ID: <200204251556.g3PFuGP15564@odiug.zope.com> > Should the use of deprecated functions in the stdlib be removed? > string.atoi -> int > string.atof -> float > sys.exc_trackback -> sys.exc_info()[2] Sure. > Should the methods which use gopher also be deprecated? Is the gopher protocol really dead? Maybe we deprecated it a little too soon; isn't it still in use at some places? --Guido van Rossum (home page: http://www.python.org/~guido/) From neal@metaslash.com Thu Apr 25 17:10:39 2002 From: neal@metaslash.com (Neal Norwitz) Date: Thu, 25 Apr 2002 12:10:39 -0400 Subject: [Python-Dev] More deprecations References: <3CC823FC.CF687436@metaslash.com> <200204251556.g3PFuGP15564@odiug.zope.com> Message-ID: <3CC82A7F.33140E6B@metaslash.com> Guido van Rossum wrote: > > Should the methods which use gopher also be deprecated? > > Is the gopher protocol really dead? Maybe we deprecated it a little > too soon; isn't it still in use at some places? gopher://gopher.floodgap.com/0/v2/vstat says it could find "555 unique servers (counting host and port pairs)." Neal From aahz@pythoncraft.com Thu Apr 25 17:14:18 2002 From: aahz@pythoncraft.com (Aahz) Date: Thu, 25 Apr 2002 12:14:18 -0400 Subject: [Python-Dev] More deprecations In-Reply-To: <200204251556.g3PFuGP15564@odiug.zope.com> References: <3CC823FC.CF687436@metaslash.com> <200204251556.g3PFuGP15564@odiug.zope.com> Message-ID: <20020425161417.GA18992@panix.com> On Thu, Apr 25, 2002, Guido van Rossum wrote: >Neal Norwitz: >> >> Should the methods which use gopher also be deprecated? > > Is the gopher protocol really dead? Maybe we deprecated it a little > too soon; isn't it still in use at some places? Yup. That's what I said last time around. I guess the deprecation still made it into the official docs. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ What if there were no rhetorical questions? From fdrake@acm.org Thu Apr 25 17:20:51 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 25 Apr 2002 12:20:51 -0400 Subject: [Python-Dev] More deprecations In-Reply-To: <20020425161417.GA18992@panix.com> References: <3CC823FC.CF687436@metaslash.com> <200204251556.g3PFuGP15564@odiug.zope.com> <20020425161417.GA18992@panix.com> Message-ID: <15560.11491.771257.339213@grendel.zope.com> Aahz writes: > Yup. That's what I said last time around. I guess the deprecation > still made it into the official docs. I mark things when I think Guido wants the deprecation. I can change the docs if we need to. I see no specific need to deprecate the gopher module. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From neal@metaslash.com Thu Apr 25 17:26:56 2002 From: neal@metaslash.com (Neal Norwitz) Date: Thu, 25 Apr 2002 12:26:56 -0400 Subject: [Python-Dev] More deprecations References: <3CC823FC.CF687436@metaslash.com> <200204251556.g3PFuGP15564@odiug.zope.com> <20020425161417.GA18992@panix.com> <15560.11491.771257.339213@grendel.zope.com> Message-ID: <3CC82E50.31C5B7FC@metaslash.com> "Fred L. Drake, Jr." wrote: > > Aahz writes: > > Yup. That's what I said last time around. I guess the deprecation > > still made it into the official docs. > > I mark things when I think Guido wants the deprecation. I can change > the docs if we need to. I see no specific need to deprecate the > gopher module. gopherlib is deprecated in PEP 4. Neal From skip@pobox.com Thu Apr 25 17:38:35 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 25 Apr 2002 11:38:35 -0500 Subject: [Python-Dev] More deprecations In-Reply-To: <3CC82A7F.33140E6B@metaslash.com> References: <3CC823FC.CF687436@metaslash.com> <200204251556.g3PFuGP15564@odiug.zope.com> <3CC82A7F.33140E6B@metaslash.com> Message-ID: <15560.12555.476522.357293@12-248-41-177.client.attbi.com> Guido> Is the gopher protocol really dead? Neal> gopher://gopher.floodgap.com/0/v2/vstat Neal> says it could find "555 unique servers (counting host and port Neal> pairs)." Recent versions of Opera don't support gopher directly. I tried clicking the above link and Opera said: The address type requires the use of a proxy server. While it may not technically be dead, it certainly appears it's pulse is weak. Skip From guido@python.org Thu Apr 25 17:40:22 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 25 Apr 2002 12:40:22 -0400 Subject: [Python-Dev] More deprecations In-Reply-To: Your message of "Thu, 25 Apr 2002 11:38:35 CDT." <15560.12555.476522.357293@12-248-41-177.client.attbi.com> References: <3CC823FC.CF687436@metaslash.com> <200204251556.g3PFuGP15564@odiug.zope.com> <3CC82A7F.33140E6B@metaslash.com> <15560.12555.476522.357293@12-248-41-177.client.attbi.com> Message-ID: <200204251640.g3PGeMe15823@odiug.zope.com> > While it may not technically be dead, it certainly appears it's pulse is > weak. "I'm not dead yet!" :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From ikjos@email.uophx.edu Thu Apr 25 18:00:20 2002 From: ikjos@email.uophx.edu (Ian Kjos) Date: Thu, 25 Apr 2002 12:00:20 -0500 Subject: [Python-Dev] Threading, scalability, and possible bug report References: <01c601c1eb44$b0e0a3a0$0dce4542@austin.rr.com> <200204240451.g3O4p8C22297@pcp742651pcs.reston01.va.comcast.net> Message-ID: <00b101c1ec7a$af4bcbf0$cc058dd8@avatartech.com> So are there any MT experts out there? I'd hardly call myself an expert without meeting someone else who studied the subject to compare with. In any case, I have a policy against foisting code of this nature on others without code review. ----- Original Message ----- From: "Guido van Rossum" To: "Ian Kjos" Cc: "Python Dev List" Sent: Tuesday, April 23, 2002 11:51 PM Subject: Re: [Python-Dev] Threading, scalability, and possible bug report > > http://www.saltillotilestore.com/morethreading.py > > Cool stuff, Ian! I'm no MT expert, but I hope we'll be able to do > something useful with this for the benefit of those who are. > > --Guido van Rossum (home page: http://www.python.org/~guido/) > From pyth@devel.trillke.net Thu Apr 25 18:22:44 2002 From: pyth@devel.trillke.net (holger krekel) Date: Thu, 25 Apr 2002 19:22:44 +0200 Subject: [Python-Dev] Threading, scalability, and possible bug report In-Reply-To: <00b101c1ec7a$af4bcbf0$cc058dd8@avatartech.com> References: <01c601c1eb44$b0e0a3a0$0dce4542@austin.rr.com> <200204240451.g3O4p8C22297@pcp742651pcs.reston01.va.comcast.net> <00b101c1ec7a$af4bcbf0$cc058dd8@avatartech.com> Message-ID: <20020425172244.GC22555@devel.trillke> On Thu, Apr 25, 2002 at 12:00:20PM -0500, Ian Kjos wrote: > So are there any MT experts out there? I'd hardly call myself an expert > without meeting someone else who studied the subject to compare with. In any > case, I have a policy against foisting code of this nature on others without > code review. good style! One one side i know some things as i have implemented a multithreaded distributed transaction service in c++ (which uses similar idioms like your code). But i don't think that my python knowledge in this area is good enough. holger From martin@v.loewis.de Thu Apr 25 18:58:10 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 25 Apr 2002 19:58:10 +0200 Subject: [Python-Dev] More deprecations In-Reply-To: <3CC823FC.CF687436@metaslash.com> References: <3CC823FC.CF687436@metaslash.com> Message-ID: Neal Norwitz writes: > Should the methods which use gopher also be deprecated? On the deprecation of gopherlib: Shortly after I wrote the initial RFC, I got some comments that gopher, as a protocol, is not dead yet, and that hence gopherlib should not be deprecated. I'd like to hear opinions whether people still think this is the case today. Regards, Martin From mal@lemburg.com Thu Apr 25 19:16:03 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 25 Apr 2002 20:16:03 +0200 Subject: [Python-Dev] More deprecations References: <3CC823FC.CF687436@metaslash.com> Message-ID: <3CC847E3.EBC47BEF@lemburg.com> "Martin v. Loewis" wrote: > > Neal Norwitz writes: > > > Should the methods which use gopher also be deprecated? > > On the deprecation of gopherlib: Shortly after I wrote the initial > RFC, I got some comments that gopher, as a protocol, is not dead yet, > and that hence gopherlib should not be deprecated. > > I'd like to hear opinions whether people still think this is the case > today. http://www.scn.org/~bkarger/gopher-manifesto gopher://gopher.browser.org/ -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From martin@v.loewis.de Thu Apr 25 19:02:11 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 25 Apr 2002 20:02:11 +0200 Subject: [Python-Dev] Threading, scalability, and possible bug report In-Reply-To: <00b101c1ec7a$af4bcbf0$cc058dd8@avatartech.com> References: <01c601c1eb44$b0e0a3a0$0dce4542@austin.rr.com> <200204240451.g3O4p8C22297@pcp742651pcs.reston01.va.comcast.net> <00b101c1ec7a$af4bcbf0$cc058dd8@avatartech.com> Message-ID: "Ian Kjos" writes: > So are there any MT experts out there? Given the limited reaction, I suggest you first try circulating this more widely. Post announcements to c.l.p.a, create mailing lists (or use existing mailing lists) etc. Regards, Martin From mal@lemburg.com Thu Apr 25 19:21:48 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 25 Apr 2002 20:21:48 +0200 Subject: [Python-Dev] unicodeobject.c,2.139,2.140 checkin References: Message-ID: <3CC8493C.DC83368C@lemburg.com> "Martin v. Loewis" wrote: > > Since this appears to be religious, I'm backing the patch out. Thanks. Please note that there's nothing religious about this: if you come up with a counting version of the codec which has similar performance or is even faster, I'd have no objections at all about your checkin. BTW, you could easily add your codec to the Python codec registry as e.g. 'utf8alt'. People could then enable it on memory starved machines by editing the codec alias list in encodings/aliases.py. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From mal@lemburg.com Thu Apr 25 19:39:27 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 25 Apr 2002 20:39:27 +0200 Subject: [Python-Dev] unicodeobject.c,2.139,2.140 checkin References: Message-ID: <3CC84D5F.32BA9085@lemburg.com> I don't know why it is, but Unicode always seems to unnecessarily heat up any discussion involving it. I would really like to know what is causing this: is it a religious issue, does it have to do with the people involved or is Unicode inherently controversial ? In any case, I wasn't looking for a fight when requesting to back out Martin's changes. Tim Peters wrote: > > [MAL] > > Tim, I don't get it... why all the fuzz about some missing asserts? > > They were requested repeatedly, and would have saved everyone hours of > debugging time. You didn't add asserts when requested, and didn't oppose > them either -- they just got ignored. Tim, I wasn't aware that you were requesting to add these. I remember that you mentioned something about using assert()s in the Python core many months ago after a discussion with Guido about this, but it wasn't clear to me that you wanted me to add assert()s to the codecs. The reason I wasn't adding assert()s is simply that I normally don't use them in C programming. This example shows that it would probably have been better to do so and I'll try to remember this for the future. > The last two times this routine was > implicated in memory corruption, you assumed it was a pymalloc bug, and > Martin had to do all the work of indentifying the true cause and then > convincing you of that. The first time that was understandable, but the > second time it wasn't; then that the third version of the code still didn't > try to catch overwrites was over the edge. True, the code was complicated due to the many different paths the codec could take. In the last version, I ripped out all the counting code because of the troubles it caused previously. In any case, before spending too much time understanding the code, why didn't you just ask for a better explanation ? I would have been the last to deny that request. > > I must have missed Barry post, sorry. > > Barry filed the latest bug report, and you closed it. The memory corruption > showed up as a bad UTF-8 translation in Mailman: > > http://www.python.org/sf/541828 True, but Martin and you continued discussing the bug after it had been closed. I hadn't read those messages, because I thought they were part of the usual SF duplication of message delivery. Sorry about that. > > I didn't leave out the asserts for any reason -- just didn't think > > about using them. > > You would have had you read the comments on the bug reports you closed, so > I'm not sure what to make of that. I'll be happy if you use asserts in the > future, when the code is getting too tricky to be obviously correct; that > shouldn't need to be a protracted battle. Will do. Again, there was no fight intended. assert(peace), -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From guido@python.org Thu Apr 25 19:59:29 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 25 Apr 2002 14:59:29 -0400 Subject: [Python-Dev] unicodeobject.c,2.139,2.140 checkin In-Reply-To: Your message of "Thu, 25 Apr 2002 20:39:27 +0200." <3CC84D5F.32BA9085@lemburg.com> References: <3CC84D5F.32BA9085@lemburg.com> Message-ID: <200204251859.g3PIxTe16275@odiug.zope.com> > I don't know why it is, but Unicode always seems to unnecessarily > heat up any discussion involving it. I would really like to know > what is causing this: is it a religious issue, does it have to do > with the people involved or is Unicode inherently controversial ? The latter definitely playes a role -- when I was going to IETF meetings back around 94/95, character set issues were always good for a few big fights, and I think the people in Asia are still not all in agreement. Another issue is that adding Unicode was probably the most invasive set of changes ever made to the Python code base. It has complicated many parts of the code, and added at least a proportional share of bugs. (I found 166 source files in CVS containing some variation on the string "unicode", and 110 bug reports mentioning "unicode" in the SF bug tracker.) For a feature that few of the developers ever need to use for themselves (I believe everyone with CVS commit privileges needs at most Latin-1 for their own language :-), I can understand that makes it a touchy issue. --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@zope.com Thu Apr 25 21:22:42 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Thu, 25 Apr 2002 16:22:42 -0400 Subject: [Python-Dev] More deprecations In-Reply-To: <3CC847E3.EBC47BEF@lemburg.com> References: <3CC823FC.CF687436@metaslash.com> <3CC847E3.EBC47BEF@lemburg.com> Message-ID: <15560.26002.457659.73576@slothrop.zope.com> >>>>> "MAL" == mal writes: MAL> "Martin v. Loewis" wrote: >> >> Neal Norwitz writes: >> >> > Should the methods which use gopher also be deprecated? >> >> On the deprecation of gopherlib: Shortly after I wrote the >> initial RFC, I got some comments that gopher, as a protocol, is >> not dead yet, and that hence gopherlib should not be deprecated. >> >> I'd like to hear opinions whether people still think this is the >> case today. MAL> http://www.scn.org/~bkarger/gopher-manifesto MAL> gopher://gopher.browser.org/ In case you haven't followed these links yet or poked around in the odd alternate universe they describe <0.4 wink>, it looks like gopher is still kicking. My Mozilla browser supports it, and there seem to be a bunch of servers out there with current information. I think gopherlib should be un-deprecated. Jeremy From Jack.Jansen@oratrix.com Thu Apr 25 22:40:51 2002 From: Jack.Jansen@oratrix.com (Jack Jansen) Date: Thu, 25 Apr 2002 23:40:51 +0200 Subject: [Python-Dev] unicodeobject.c,2.139,2.140 checkin In-Reply-To: <200204251859.g3PIxTe16275@odiug.zope.com> Message-ID: <1D0866D4-5895-11D6-A56C-003065517236@oratrix.com> On donderdag, april 25, 2002, at 08:59 , Guido van Rossum wrote: >> I don't know why it is, but Unicode always seems to unnecessarily >> heat up any discussion involving it. I would really like to know >> what is causing this: is it a religious issue, does it have to do >> with the people involved or is Unicode inherently controversial ? > [...] > Another issue is that adding Unicode was probably the most invasive > set of changes ever made to the Python code base. It has complicated > many parts of the code, and added at least a proportional share of > bugs. (I found 166 source files in CVS containing some variation on > the string "unicode", and 110 bug reports mentioning "unicode" in the > SF bug tracker.) Another thing that bothers me is that it retroactively changed the interpretation of other Python objects. For me it's perfectly logical that a character string is a character string, unless there's a very good reason to treat it differently (a framebuffer scanline, a binary blob, etc). And so if I have an API OpenFileWithUnicodeName() that accepts a unicode filename I expect that if I pass an 8-bit filename it would be converted on the fly. Other people focus on different sets of API's, however, and think there's nothing more logical than interpreting the string object as a binary buffer containing UTF16 values or what-have-you. Scanlines or binary blobs hardly ever mixed with filenames, so there wasn't an issue before unicode raised its pretty/ugly head. (of course it could be argued that unicode has demonstrated a design flaw in Python, namely that a single data-type was used to store both binary data of unknown interpretation and character arrays, and that there's now little more to be done about that). -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From tim.one@comcast.net Thu Apr 25 23:25:33 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 25 Apr 2002 18:25:33 -0400 Subject: [Python-Dev] unicodeobject.c,2.139,2.140 checkin In-Reply-To: <3CC84D5F.32BA9085@lemburg.com> Message-ID: [M.-A. Lemburg] > I don't know why it is, but Unicode always seems to unnecessarily > heat up any discussion involving it. Huh -- I thought I was the only one who noticed this . > I would really like to know what is causing this: is it a religious > issue, does it have to do with the people involved or is Unicode > inherently controversial ? Unicode had nothing to do with my yelling in this thread. I've got very low tolerance for memory corruption, regardless of source. When it happens once I'm on high alert, when it happens twice in the same place I go postal. Had this been in dictobject.c or boolobject.c, I would have been just as unhappy. Now that the memory corruption is thought to be solved, and verified in the debug build regardless, *now* I'll get cranky about foreigners and their lameass character sets . On the technical issues remaining, I don't know how to judge the tradeoff between memory use and speed here. If you do, and pymalloc can help in some way, I'll be happy to help. From barry@zope.com Thu Apr 25 23:46:01 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 25 Apr 2002 18:46:01 -0400 Subject: [Python-Dev] Fun with numbers Message-ID: <15560.34601.661364.356755@anthem.wooz.org> Tim, Jeremy, and I are having so much fun... :) -Barry -------------------- snip snip -------------------- Python 2.3a0 (#14, Apr 23 2002, 18:26:18) [GCC 2.96 20000731 (Mandrake Linux 8.2 2.96-0.76mdk)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 3 .__class__ [9903 refs] >>> 3 .__class__ = int [9903 refs] >>> 3 .__class__ = float Traceback (most recent call last): File "", line 1, in ? TypeError: __class__ assignment: 'float' object layout differs from 'int' [9939 refs] >>> 3 .__class__ = bool [9939 refs] >>> 3 True [9939 refs] >>> 3L .__class__ = int Traceback (most recent call last): File "", line 1, in ? TypeError: 'long' object has only read-only attributes (assign to .__class__) [9939 refs] >>> 23000 .__class__ = bool Debug memory block at address p=0x814f544: 485823496 bytes originally requested the 4 pad bytes at p-4 are not all FORBIDDENBYTE (0xfb): at p-4: 0x7a *** OUCH at p-3: 0x61 *** OUCH at p-2: 0xc8 *** OUCH at p-1: 0x3c *** OUCH the 4 pad bytes at tail=0x250a094c are Segmentation fault From skip@pobox.com Fri Apr 26 00:32:13 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 25 Apr 2002 18:32:13 -0500 Subject: [Python-Dev] More deprecations In-Reply-To: <15560.26002.457659.73576@slothrop.zope.com> References: <3CC823FC.CF687436@metaslash.com> <3CC847E3.EBC47BEF@lemburg.com> <15560.26002.457659.73576@slothrop.zope.com> Message-ID: <15560.37373.394560.447941@12-248-41-177.client.attbi.com> Jeremy> In case you haven't followed these links yet or poked around in Jeremy> the odd alternate universe they describe <0.4 wink>, it looks Jeremy> like gopher is still kicking. It appears there is a Python-based gopher client. From the gopher:// manifesto: Hot new gopher client (Python package!): http://opop.nols.com/proggie/forg-latest.tar.gz Looks like the most recent update was last September. I tried to run it but it requires Pmw. Skip From guido@python.org Fri Apr 26 02:02:53 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 25 Apr 2002 21:02:53 -0400 Subject: [Python-Dev] Fun with numbers In-Reply-To: Your message of "Thu, 25 Apr 2002 18:46:01 EDT." <15560.34601.661364.356755@anthem.wooz.org> References: <15560.34601.661364.356755@anthem.wooz.org> Message-ID: <200204260102.g3Q12r927686@pcp742651pcs.reston01.va.comcast.net> > Tim, Jeremy, and I are having so much fun... [...] > >>> 23000 .__class__ = bool > Debug memory block at address p=0x814f544: > 485823496 bytes originally requested > the 4 pad bytes at p-4 are not all FORBIDDENBYTE (0xfb): > at p-4: 0x7a *** OUCH > at p-3: 0x61 *** OUCH > at p-2: 0xc8 *** OUCH > at p-1: 0x3c *** OUCH > the 4 pad bytes at tail=0x250a094c are Segmentation fault Ouch. Fixed in CVS for this particular case, but I think you may still be able provoke this with a built-in type that derives from another built-in type without adding any new fields, if the base type has a funky allocator that the derived type doesn't inherit. Also, the fix means that if an extension defines a type that inherits from int but doesn't override tp_alloc and tp_free, it inherits a tp_alloc that's not matched to the tp_free. But why would anyone do that...? Inspired by this, I tried something else. >>> print True True >>> True.__class__ = int >>> print True 1 >>> There are no ill side effects. Handy for bool-haters. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Fri Apr 26 03:30:48 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 25 Apr 2002 22:30:48 -0400 Subject: [Python-Dev] More deprecations In-Reply-To: <3CC823FC.CF687436@metaslash.com> References: <3CC823FC.CF687436@metaslash.com> Message-ID: <15560.48088.698840.716694@grendel.zope.com> Neal Norwitz writes: > Should the use of deprecated functions in the stdlib be removed? > string.atoi -> int > string.atof -> float > sys.exc_trackback -> sys.exc_info()[2] Yes, especially that last one, since there are thread safety issues. > ConfigParser.py:311: string.atoi is deprecated > ConfigParser.py:314: string.atof is deprecated > Cookie.py:331: string.atoi is deprecated > inspect.py:658: sys.exc_traceback is deprecated These are now fixed in CVS. Thanks! -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From neal@metaslash.com Fri Apr 26 04:03:41 2002 From: neal@metaslash.com (Neal Norwitz) Date: Thu, 25 Apr 2002 23:03:41 -0400 Subject: [Python-Dev] More deprecations References: <3CC823FC.CF687436@metaslash.com> <15560.48088.698840.716694@grendel.zope.com> Message-ID: <3CC8C38D.30009F83@metaslash.com> "Fred L. Drake, Jr." wrote: > > Neal Norwitz writes: > > Should the use of deprecated functions in the stdlib be removed? > > string.atoi -> int > > string.atof -> float > > sys.exc_trackback -> sys.exc_info()[2] > > Yes, especially that last one, since there are thread safety issues. > > > ConfigParser.py:311: string.atoi is deprecated > > ConfigParser.py:314: string.atof is deprecated > > Cookie.py:331: string.atoi is deprecated > > inspect.py:658: sys.exc_traceback is deprecated > > These are now fixed in CVS. Thanks! Great! Thanks, you saved me the work. I noticed your changes to use string methods. Should I change other uses? I've got diffs for: ftplib, markupbase, and tabnanny. Should I check those in? What are the modules that should keep backwards compatability with 1.5.2? It seems these are currently: inspect pydoc sre* xmlrpclib Is this correct? Are there others? Neal From guido@python.org Fri Apr 26 04:08:37 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 25 Apr 2002 23:08:37 -0400 Subject: [Python-Dev] Making None a keyword Message-ID: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> I'd like to make None a keyword. This prevents dumb users from assigning to it and screwing themselves, and can cause a slight speedup because using None avoids two dict lookups. - Any objections? - Can somebody help me implement this? I've got the parser changes ready, but not the compiler changes. Believe it or not, Zope3 contains code that will break with this change: there are functions with a default argument of the form None=None as a speedup hack. I think this is an argument *for* the change. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@zope.com Fri Apr 26 04:08:16 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Thu, 25 Apr 2002 23:08:16 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> References: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15560.50336.965410.772225@slothrop.zope.com> I can help with the compiler changes. What about True and False? Same logic -- prevent accidental harm, prevent people from doing silly things to make them faster. Jeremy From guido@python.org Fri Apr 26 04:17:35 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 25 Apr 2002 23:17:35 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: Your message of "Thu, 25 Apr 2002 23:08:16 EDT." <15560.50336.965410.772225@slothrop.zope.com> References: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> <15560.50336.965410.772225@slothrop.zope.com> Message-ID: <200204260317.g3Q3HZQ32684@pcp742651pcs.reston01.va.comcast.net> > I can help with the compiler changes. Great! (Though I was hoping someone from outside PythonLabs -- we're already too busy, and you in particular.) > What about True and False? Same logic -- prevent accidental harm, > prevent people from doing silly things to make them faster. Maybe in Python 3.0. Old code could define these as variables and we don't want to break that. (I have no mercy for old code that defines None, but True and False are a difference case. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Fri Apr 26 04:30:08 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 25 Apr 2002 23:30:08 -0400 Subject: [Python-Dev] Making None a keyword References: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15560.51648.962067.387651@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> I'd like to make None a keyword. Now, how did I know you were going to want to do that? From python-mode.el (for font-locking): ;; pseudo-keywords '("\\b\\(self\\|None\\|True\\|False\\|Ellipsis\\)\\b" I swear I filled the time machine's tank when I was done with it. +1 BTW. >>>>> "JH" == Jeremy Hylton writes: JH> What about True and False? GvR> Maybe in Python 3.0. Old code could define these as GvR> variables and we don't want to break that. (I have no mercy GvR> for old code that defines None, but True and False are a GvR> difference case. :-) Ellipsis then? It's probably used /much/ less than None, but still. -Barry From guido@python.org Fri Apr 26 04:35:36 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 25 Apr 2002 23:35:36 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: Your message of "Thu, 25 Apr 2002 23:30:08 EDT." <15560.51648.962067.387651@anthem.wooz.org> References: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> <15560.51648.962067.387651@anthem.wooz.org> Message-ID: <200204260335.g3Q3Zap00890@pcp742651pcs.reston01.va.comcast.net> > Ellipsis then? It's probably used /much/ less than None, but still. Not worth it (it's a fairly big hack in the compiler). --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@zope.com Fri Apr 26 04:43:17 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Thu, 25 Apr 2002 23:43:17 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: <200204260317.g3Q3HZQ32684@pcp742651pcs.reston01.va.comcast.net> References: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> <15560.50336.965410.772225@slothrop.zope.com> <200204260317.g3Q3HZQ32684@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15560.52437.47576.481530@slothrop.zope.com> >>>>> "GvR" == Guido van Rossum writes: >> I can help with the compiler changes. GvR> Great! (Though I was hoping someone from outside PythonLabs -- GvR> we're already too busy, and you in particular.) If someone else wants to do it, I'll be happy to help. I think it's a small project. We've got a new node type in the grammar -- None -- and we'll need a new opcode -- LOAD_NONE. The compiler just needs to generate a LOAD_NONE for every None node. Does the parser prevent assignment to None? Or does the compiler need to check for the use of None in an assignment? Jeremy From guido@python.org Fri Apr 26 04:53:44 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 25 Apr 2002 23:53:44 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: Your message of "Thu, 25 Apr 2002 23:43:17 EDT." <15560.52437.47576.481530@slothrop.zope.com> References: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> <15560.50336.965410.772225@slothrop.zope.com> <200204260317.g3Q3HZQ32684@pcp742651pcs.reston01.va.comcast.net> <15560.52437.47576.481530@slothrop.zope.com> Message-ID: <200204260353.g3Q3rjc01362@pcp742651pcs.reston01.va.comcast.net> > I think it's a small project. We've got a new node type in the > grammar -- None -- and we'll need a new opcode -- LOAD_NONE. The > compiler just needs to generate a LOAD_NONE for every None node. > > Does the parser prevent assignment to None? Or does the compiler need > to check for the use of None in an assignment? The compiler needs to check. It's just like assigning to a literal: the parser has no power to prevent this, so the compiler has to act as a safeguard. --Guido van Rossum (home page: http://www.python.org/~guido/) From nas@python.ca Fri Apr 26 05:06:31 2002 From: nas@python.ca (Neil Schemenauer) Date: Thu, 25 Apr 2002 21:06:31 -0700 Subject: [Python-Dev] Making None a keyword In-Reply-To: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Thu, Apr 25, 2002 at 11:08:37PM -0400 References: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020425210631.A2868@glacier.arctrix.com> Guido van Rossum wrote: > - Can somebody help me implement this? I've got the parser changes > ready, but not the compiler changes. Where are the parser changes? I'm willing to give it a shot. Neil From niemeyer@conectiva.com Fri Apr 26 05:04:36 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Fri, 26 Apr 2002 01:04:36 -0300 Subject: [Python-Dev] Making None a keyword In-Reply-To: <15560.52437.47576.481530@slothrop.zope.com> References: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> <15560.50336.965410.772225@slothrop.zope.com> <200204260317.g3Q3HZQ32684@pcp742651pcs.reston01.va.comcast.net> <15560.52437.47576.481530@slothrop.zope.com> Message-ID: <20020426010436.A5770@ibook> > If someone else wants to do it, I'll be happy to help. > > I think it's a small project. We've got a new node type in the > grammar -- None -- and we'll need a new opcode -- LOAD_NONE. The > compiler just needs to generate a LOAD_NONE for every None node. I have hacked the interpreter loop before, but haven't changed the compiler yet. It's a nice oportunity to play with it, so I'll be glad to help, and accept your help as well. I'm going to study it more deeply tomorrow. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From akuchlin@mems-exchange.org Fri Apr 26 05:38:58 2002 From: akuchlin@mems-exchange.org (akuchlin@mems-exchange.org) Date: Fri, 26 Apr 2002 00:38:58 -0400 Subject: [Python-Dev] More deprecations In-Reply-To: <3CC8C38D.30009F83@metaslash.com>; from neal@metaslash.com on Thu, Apr 25, 2002 at 11:03:41PM -0400 References: <3CC823FC.CF687436@metaslash.com> <15560.48088.698840.716694@grendel.zope.com> <3CC8C38D.30009F83@metaslash.com> Message-ID: <20020426003858.A29630@mems-exchange.org> On Thu, Apr 25, 2002 at 11:03:41PM -0400, Neal Norwitz wrote: >What are the modules that should keep backwards compatability >with 1.5.2? It seems these are currently: Add distutils/* to the list, too. I have no plans to issue future standalone releases for 1.5.2, but Greg Ward or someone else might. --amk From fdrake@acm.org Fri Apr 26 06:16:38 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 26 Apr 2002 01:16:38 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: <15560.52437.47576.481530@slothrop.zope.com> References: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> <15560.50336.965410.772225@slothrop.zope.com> <200204260317.g3Q3HZQ32684@pcp742651pcs.reston01.va.comcast.net> <15560.52437.47576.481530@slothrop.zope.com> Message-ID: <15560.58038.737344.840187@grendel.zope.com> Jeremy Hylton writes: > Does the parser prevent assignment to None? Or does the compiler need > to check for the use of None in an assignment? Inside functions, you don't even need that. You just need to use LOAD_CONST (or whatever its called now), since None is entry 0 in the constants table, used by the implied "return None" when control drops off the end. Doing this in function buys most of the performance. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From tim.one@comcast.net Fri Apr 26 06:43:01 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 26 Apr 2002 01:43:01 -0400 Subject: [Python-Dev] More deprecations In-Reply-To: <3CC8C38D.30009F83@metaslash.com> Message-ID: [Neal Norwitz] > .. > What are the modules that should keep backwards compatability > with 1.5.2? It seems these are currently: > > inspect > pydoc The current versions of both those require 2.2. I didn't go out of my way to break 1.5.2 compatability when changing them for 2.2, but neither did I go out of my way to bury uses of new-in-2.2 staticmethod etc under obfuscating conditionals. > sre* Yes, I believe Fredrik wants compatability there. > xmlrpclib Dunno. From Oleg Broytmann Fri Apr 26 08:44:22 2002 From: Oleg Broytmann (Oleg Broytmann) Date: Fri, 26 Apr 2002 11:44:22 +0400 Subject: [Python-Dev] email, rfc 2231 Message-ID: <20020426114422.A24431@phd.pp.ru> Barry, email package in Python CVS significantly differs from the mimelib version. Shoould I wait while you update it or should I recreate my patches against Python CVS version? Oleg. -- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From mal@lemburg.com Fri Apr 26 08:52:50 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 26 Apr 2002 09:52:50 +0200 Subject: [Python-Dev] More deprecations References: <3CC823FC.CF687436@metaslash.com> <15560.48088.698840.716694@grendel.zope.com> <3CC8C38D.30009F83@metaslash.com> Message-ID: <3CC90752.82AE82A4@lemburg.com> Neal Norwitz wrote: > > What are the modules that should keep backwards compatability > with 1.5.2? It seems these are currently: In general, I think it's a good idea to be very careful about introduce flashy new stuff to the std lib since it makes backporting fixes harder, e.g. using Python 2.1 features is OK, but not 2.2 features (if not absolutely necessary or only if they are easily separatable from the rest of the code). > inspect > pydoc +0 on those > sre* > xmlrpclib +1 on those two > Is this correct? Are there others? I suppose you should add the PyXML stuff (the part which is the std dist). Don't whether the PyXML team needs these bits for their standalone distributions, though. And distutils, as Andrew already mentioned. You should also keep in mind that porting to Jython should remain easily possible. Breaking that link would cause unnecessary work on the Jython side. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From mal@lemburg.com Fri Apr 26 08:58:36 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 26 Apr 2002 09:58:36 +0200 Subject: [Python-Dev] Making None a keyword References: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CC908AC.98714272@lemburg.com> Guido van Rossum wrote: > > I'd like to make None a keyword. This prevents dumb users from > assigning to it and screwing themselves, and can cause a slight > speedup because using None avoids two dict lookups. > > - Any objections? +1 but only if you use the standard mechanism for this: warnings in 2.3, SyntaxError in 2.4. > - Can somebody help me implement this? I've got the parser changes > ready, but not the compiler changes. > > Believe it or not, Zope3 contains code that will break with this > change: there are functions with a default argument of the form > None=None as a speedup hack. I think this is an argument *for* the > change. :-) Zope3 is not the only software using these kind of hacks to work around the builtins lookups. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From mal@lemburg.com Fri Apr 26 09:26:22 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 26 Apr 2002 10:26:22 +0200 Subject: [Python-Dev] pymalloc and overallocation (unicodeobject.c,2.139,2.140 checkin) References: Message-ID: <3CC90F2E.8235A5EA@lemburg.com> Tim Peters wrote: > > [M.-A. Lemburg] > > I don't know why it is, but Unicode always seems to unnecessarily > > heat up any discussion involving it. > > Huh -- I thought I was the only one who noticed this . Naa, it's occurred to me several times in the past. Unicode seems to trigger some memory corruption in Brain 2.2 which results in spilling out huge amounts of adrenalin and causes the blood pressure to reach record highs ;-) > > I would really like to know what is causing this: is it a religious > > issue, does it have to do with the people involved or is Unicode > > inherently controversial ? > > Unicode had nothing to do with my yelling in this thread. I've got very low > tolerance for memory corruption, regardless of source. When it happens once > I'm on high alert, when it happens twice in the same place I go postal. Had > this been in dictobject.c or boolobject.c, I would have been just as > unhappy. > > Now that the memory corruption is thought to be solved, and verified in the > debug build regardless, *now* I'll get cranky about foreigners and their > lameass character sets . Good to know. > On the technical issues remaining, I don't know how to judge the tradeoff > between memory use and speed here. If you do, and pymalloc can help in some > way, I'll be happy to help. First of all, UTF-8 is probably the most common Unicode encoding used today and will certainly become *the* standard encoding within the next few years. So speed matters a lot in this particular corner of the Unicode implementation. The standard reasoning behind using overallocation for memory management is that typical modern malloc()s don't really allocate the memory until it is used (you know this anyway...), so overallocation doesn't actually cause bundles of memory chips to heat up. This makes overallocation ideal for the case where you don't know the exact size in advance but where you can estimate a reasonable upper bound. Now with pymalloc the situation is a bit different for smaller sized memory areas (larger chunks are handed off to the system malloc() which uses the above strategy). As Martin's benchmark showed, the counting strategy is faster for small chunks and this is probably due to the fact that pymalloc manages these. Since pymalloc cannot know that an algorithm wants to use overallocation as memory allocation strategy, it would probably help to find a way to tell pymalloc about this fact. It could then either redirect the request to the system malloc() or use a different malloc strategy for these chunks. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From mal@lemburg.com Fri Apr 26 09:31:07 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 26 Apr 2002 10:31:07 +0200 Subject: [Python-Dev] unicodeobject.c,2.139,2.140 checkin References: <1D0866D4-5895-11D6-A56C-003065517236@oratrix.com> Message-ID: <3CC9104B.FC713BD1@lemburg.com> Jack Jansen wrote: > > (of course it could be argued that unicode has demonstrated a > design flaw in Python, namely that a single data-type was used > to store both binary data of unknown interpretation and > character arrays, and that there's now little more to be done > about that). This is probably the most significant problem with the usage of strings -- and there's nothing much we can do about it since it's hard-wired in the programmers mind... most other languages have the same problem. In the long run, I think that the Unicode type should be used for all text data and strings for binary data of unknown encoding. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From Anthony Baxter Fri Apr 26 09:30:07 2002 From: Anthony Baxter (Anthony Baxter) Date: Fri, 26 Apr 2002 18:30:07 +1000 Subject: [Python-Dev] METH_CLASS In-Reply-To: Message from Guido van Rossum of "Thu, 25 Apr 2002 08:59:39 -0400." <200204251259.g3PCxdk25615@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204260830.g3Q8U7l13361@localhost.localdomain> >>> Guido van Rossum wrote > > 2. Since this is marked new in 2.3: Is it planned to backport > > this stuff into the 2.2.x series? > > Requiring Python 2.3 for my stuff seems a but harsh currently... > > I don't plan to backport this to 2.2 unless there's an overwhelming > demand. I'd suggest that this _isn't_ the sort of thing that should be going into 2.2. Aside from the amount of work - I'm guessing it'd be a bit - the notion of adding stuff like this to a maintenance/bugfix release seems, to me, to be stretching the meaning of the term maintenance release. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From mal@lemburg.com Fri Apr 26 09:49:04 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 26 Apr 2002 10:49:04 +0200 Subject: [Python-Dev] Unicode References: <3CC84D5F.32BA9085@lemburg.com> <200204251859.g3PIxTe16275@odiug.zope.com> Message-ID: <3CC91480.DF8A46ED@lemburg.com> Guido van Rossum wrote: > > > I don't know why it is, but Unicode always seems to unnecessarily > > heat up any discussion involving it. I would really like to know > > what is causing this: is it a religious issue, does it have to do > > with the people involved or is Unicode inherently controversial ? > > The latter definitely playes a role -- when I was going to IETF > meetings back around 94/95, character set issues were always good for > a few big fights, and I think the people in Asia are still not all in > agreement. > > Another issue is that adding Unicode was probably the most invasive > set of changes ever made to the Python code base. It has complicated > many parts of the code, and added at least a proportional share of > bugs. (I found 166 source files in CVS containing some variation on > the string "unicode", and 110 bug reports mentioning "unicode" in the > SF bug tracker.) True; and it was hard enough to get it mostly to a working compromise. > For a feature that few of the developers ever need to use for > themselves (I believe everyone with CVS commit privileges needs at > most Latin-1 for their own language :-), I can understand that makes > it a touchy issue. Very true indeed. Still, I think Unicode gives a chance of "fixing" the problem we currently have with strings: Unicode is unlike strings only usable for text data and that makes it ideal as standard type for text -- we'll never convince people to make a difference between text and binary data in strings, so offering them Unicode as alternative is a good strategy, IMHO. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From mwh@python.net Fri Apr 26 11:40:02 2002 From: mwh@python.net (Michael Hudson) Date: 26 Apr 2002 11:40:02 +0100 Subject: [Python-Dev] Making None a keyword In-Reply-To: Guido van Rossum's message of "Thu, 25 Apr 2002 23:08:37 -0400" References: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> Message-ID: <2m7kmurbel.fsf@starship.python.net> Guido van Rossum writes: > I'd like to make None a keyword. This prevents dumb users from > assigning to it and screwing themselves, and can cause a slight > speedup because using None avoids two dict lookups. You are going to make this issue a warning in 2.3, right? Cheers, M. -- There are two ways of constructing a software design: one way is to make it so simple that there are obviously no deficiencies and the other way is to make it so complicated that there are no obvious deficiencies. -- C. A. R. Hoare From guido@python.org Fri Apr 26 13:10:24 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 08:10:24 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: Your message of "Fri, 26 Apr 2002 09:58:36 +0200." <3CC908AC.98714272@lemburg.com> References: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> <3CC908AC.98714272@lemburg.com> Message-ID: <200204261210.g3QCAP501999@pcp742651pcs.reston01.va.comcast.net> > > I'd like to make None a keyword. This prevents dumb users from > > assigning to it and screwing themselves, and can cause a slight > > speedup because using None avoids two dict lookups. > > > > - Any objections? > > +1 but only if you use the standard mechanism for this: > warnings in 2.3, SyntaxError in 2.4. Good point. > > - Can somebody help me implement this? I've got the parser changes > > ready, but not the compiler changes. > > > > Believe it or not, Zope3 contains code that will break with this > > change: there are functions with a default argument of the form > > None=None as a speedup hack. I think this is an argument *for* the > > change. :-) > > Zope3 is not the only software using these kind of hacks > to work around the builtins lookups. I just realized there's another use for None as an identifier that is currently totally legal and which would become illegal: it's conceivable that someone would use None as an attribute name, e.g. class C: def None(self): pass C().None() --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Apr 26 13:19:38 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 08:19:38 -0400 Subject: [Python-Dev] unicodeobject.c,2.139,2.140 checkin In-Reply-To: Your message of "Fri, 26 Apr 2002 10:31:07 +0200." <3CC9104B.FC713BD1@lemburg.com> References: <1D0866D4-5895-11D6-A56C-003065517236@oratrix.com> <3CC9104B.FC713BD1@lemburg.com> Message-ID: <200204261219.g3QCJci02177@pcp742651pcs.reston01.va.comcast.net> > In the long run, I think that the Unicode type should be > used for all text data and strings for binary data of unknown > encoding. I think you're underestimating the tenacity of ASCII and Latin-1. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Apr 26 13:23:51 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 08:23:51 -0400 Subject: [Python-Dev] Unicode In-Reply-To: Your message of "Fri, 26 Apr 2002 10:49:04 +0200." <3CC91480.DF8A46ED@lemburg.com> References: <3CC84D5F.32BA9085@lemburg.com> <200204251859.g3PIxTe16275@odiug.zope.com> <3CC91480.DF8A46ED@lemburg.com> Message-ID: <200204261223.g3QCNpN02195@pcp742651pcs.reston01.va.comcast.net> > True; and it was hard enough to get it mostly to a working > compromise. Let me add that I very much appreciate your heroic efforts there!!! > Very true indeed. Still, I think Unicode gives a chance of > "fixing" the problem we currently have with strings: Unicode > is unlike strings only usable for text data and that makes > it ideal as standard type for text -- we'll never convince > people to make a difference between text and binary data in > strings, so offering them Unicode as alternative is a good > strategy, IMHO. It's a long way before we're there though -- we'd have to overhaul the I/O system entirely, and that takes a lot of time not just because of the effort but also because it won't be fully compatible. Also, once 8-bit strings are used for binary data only, I wonder if they shouldn't be more like Java's byte arrays -- i.e. mutable. And they don't need a literal notation. That's another major language change. :-( --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Apr 26 13:42:56 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 08:42:56 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: Your message of "Fri, 26 Apr 2002 01:04:36 -0300." <20020426010436.A5770@ibook> References: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> <15560.50336.965410.772225@slothrop.zope.com> <200204260317.g3Q3HZQ32684@pcp742651pcs.reston01.va.comcast.net> <15560.52437.47576.481530@slothrop.zope.com> <20020426010436.A5770@ibook> Message-ID: <200204261242.g3QCguQ02366@pcp742651pcs.reston01.va.comcast.net> Here's what I have so far. :-) *** Grammar/Grammar 15 Oct 2001 15:44:04 -0000 1.45 --- Grammar/Grammar 26 Apr 2002 12:30:36 -0000 *************** *** 80,86 **** term: factor (('*'|'/'|'%'|'//') factor)* factor: ('+'|'-'|'~') factor | power power: atom trailer* ('**' factor)* ! atom: '(' [testlist] ')' | '[' [listmaker] ']' | '{' [dictmaker] '}' | '`' testlist '`' | NAME | NUMBER | STRING+ listmaker: test ( list_for | (',' test)* [','] ) lambdef: 'lambda' [varargslist] ':' test trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME --- 80,86 ---- term: factor (('*'|'/'|'%'|'//') factor)* factor: ('+'|'-'|'~') factor | power power: atom trailer* ('**' factor)* ! atom: '(' [testlist] ')' | '[' [listmaker] ']' | '{' [dictmaker] '}' | '`' testlist '`' | NAME | NUMBER | STRING+ | 'None' listmaker: test ( list_for | (',' test)* [','] ) lambdef: 'lambda' [varargslist] ':' test trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME It needs to add 'None' as an alternative to all other places where currently NAME occurs. The Unix Makefile automatically regenerates Includes/graminit.h and Python/graminit.c when you touch Grammar/Grammar. For some mysterious reason, I didn't have to change the code generator at all -- the NAME terminals and the 'None' terminal look the same to it. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Fri Apr 26 13:42:48 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 26 Apr 2002 08:42:48 -0400 Subject: [Python-Dev] Re: email, rfc 2231 References: <20020426114422.A24431@phd.pp.ru> Message-ID: <15561.19272.259210.135700@anthem.wooz.org> >>>>> "OB" == Oleg Broytmann writes: OB> email package in Python CVS significantly differs from the OB> mimelib version. Shoould I wait while you update it or should OB> I recreate my patches against Python CVS version? I'm not planning on updating the mimelib version any time soon. What I might do is create a branch in the Python cvs so that I can backport it to Python 2.1. Or maybe I should just backport what's in the cvs right now (it's not much -- there are a couple of generators there that need to be rewritten). A patch against Python CVS would be best. -Barry From barry@zope.com Fri Apr 26 13:45:12 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 26 Apr 2002 08:45:12 -0400 Subject: [Python-Dev] More deprecations References: <3CC823FC.CF687436@metaslash.com> <15560.48088.698840.716694@grendel.zope.com> <3CC8C38D.30009F83@metaslash.com> <3CC90752.82AE82A4@lemburg.com> Message-ID: <15561.19416.407233.566080@anthem.wooz.org> >>>>> "MAL" == M writes: MAL> In general, I think it's a good idea to be very careful about MAL> introduce flashy new stuff to the std lib since it makes MAL> backporting fixes harder, e.g. using Python 2.1 features is MAL> OK, but not 2.2 features (if not absolutely necessary or only MAL> if they are easily separatable from the rest of the code). This is emerging as a common theme, and a good one to adhere to. I wonder if we should formalize it (i.e. target the stdlib to Python 2.1 where possible)? -Barry From barry@zope.com Fri Apr 26 13:51:14 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 26 Apr 2002 08:51:14 -0400 Subject: [Python-Dev] Making None a keyword References: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> <3CC908AC.98714272@lemburg.com> <200204261210.g3QCAP501999@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15561.19778.901038.782154@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> I just realized there's another use for None as an identifier GvR> that is currently totally legal and which would become GvR> illegal: it's conceivable that someone would use None as an GvR> attribute name, e.g. | class C: | def None(self): pass GvR> C().None() That reminds me, I wonder if we should dust off the old Jython idea of allowing keywords in locations where their non-keywordness is obvious? C().print()-ly y'rs, -Barry From guido@python.org Fri Apr 26 13:55:35 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 08:55:35 -0400 Subject: [Python-Dev] More deprecations In-Reply-To: Your message of "Fri, 26 Apr 2002 08:45:12 EDT." <15561.19416.407233.566080@anthem.wooz.org> References: <3CC823FC.CF687436@metaslash.com> <15560.48088.698840.716694@grendel.zope.com> <3CC8C38D.30009F83@metaslash.com> <3CC90752.82AE82A4@lemburg.com> <15561.19416.407233.566080@anthem.wooz.org> Message-ID: <200204261255.g3QCtZe02428@pcp742651pcs.reston01.va.comcast.net> > This is emerging as a common theme, and a good one to adhere to. I > wonder if we should formalize it (i.e. target the stdlib to Python 2.1 > where possible)? What would the point be? How many people are moving stdlib modules to previous releases? I think very few. I just added True/False to a whole bunch of places, and I'm sure with each new enticing feature it will be harder to keep the stdlib 2.1 compatible. I'm all for marking certain modules as "have to be backwards compatible" though -- e.g. sre. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Apr 26 14:04:43 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 09:04:43 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: Your message of "Fri, 26 Apr 2002 08:51:14 EDT." <15561.19778.901038.782154@anthem.wooz.org> References: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> <3CC908AC.98714272@lemburg.com> <200204261210.g3QCAP501999@pcp742651pcs.reston01.va.comcast.net> <15561.19778.901038.782154@anthem.wooz.org> Message-ID: <200204261304.g3QD4hZ02460@pcp742651pcs.reston01.va.comcast.net> > That reminds me, I wonder if we should dust off the old Jython idea of > allowing keywords in locations where their non-keywordness is obvious? I'd rather not. --Guido van Rossum (home page: http://www.python.org/~guido/) From Oleg Broytmann Fri Apr 26 14:31:41 2002 From: Oleg Broytmann (Oleg Broytmann) Date: Fri, 26 Apr 2002 17:31:41 +0400 Subject: [Python-Dev] Re: email, rfc 2231 In-Reply-To: <15561.19272.259210.135700@anthem.wooz.org>; from barry@zope.com on Fri, Apr 26, 2002 at 08:42:48AM -0400 References: <20020426114422.A24431@phd.pp.ru> <15561.19272.259210.135700@anthem.wooz.org> Message-ID: <20020426173141.B28624@phd.pp.ru> On Fri, Apr 26, 2002 at 08:42:48AM -0400, Barry A. Warsaw wrote: > A patch against Python CVS would be best. Ok, I see. Will try to recreate patches and tests. Oleg. -- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From jeremy@ZOPE.COM Fri Apr 26 15:10:12 2002 From: jeremy@ZOPE.COM (Jeremy Hylton) Date: Fri, 26 Apr 2002 10:10:12 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: <15560.58038.737344.840187@grendel.zope.com> References: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> <15560.50336.965410.772225@slothrop.zope.com> <200204260317.g3Q3HZQ32684@pcp742651pcs.reston01.va.comcast.net> <15560.52437.47576.481530@slothrop.zope.com> <15560.58038.737344.840187@grendel.zope.com> Message-ID: <15561.24516.417362.240364@slothrop.zope.com> >>>>> "FLD" == Fred L Drake, writes: FLD> Jeremy Hylton writes: >> Does the parser prevent assignment to None? Or does the compiler >> need to check for the use of None in an assignment? FLD> Inside functions, you don't even need that. You just need to FLD> use LOAD_CONST (or whatever its called now), since None is FLD> entry 0 in the constants table, used by the implied "return FLD> None" when control drops off the end. Doing this in function FLD> buys most of the performance. I don't understand what you mean, but I'll try to reply anyway :-). I assume LOAD_NONE will eliminate the need for LOAD_CONST 0 (None). Instead of this: case LOAD_CONST: x = GETCONST(oparg); Py_INCREF(x); PUSH(x); goto fast_next_opcode; we'd have this: case LOAD_NONE: Py_INCREF(Py_None); PUSH(Py_None); goto fast_next_opcode; It's probably a wee bit faster and it makes the bytecode smaller, because you don't need None in co_consts and you don't need an argument to the bytecode. Based on my cycle counter measurements before the conference, I suspect the performance impact is, well, negligible. Jeremy From fdrake@acm.org Fri Apr 26 15:24:04 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 26 Apr 2002 10:24:04 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: <15561.24516.417362.240364@slothrop.zope.com> References: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> <15560.50336.965410.772225@slothrop.zope.com> <200204260317.g3Q3HZQ32684@pcp742651pcs.reston01.va.comcast.net> <15560.52437.47576.481530@slothrop.zope.com> <15560.58038.737344.840187@grendel.zope.com> <15561.24516.417362.240364@slothrop.zope.com> Message-ID: <15561.25348.496531.621413@grendel.zope.com> Jeremy Hylton writes: > I don't understand what you mean, but I'll try to reply anyway :-). Often a good tactic. ;-) > I assume LOAD_NONE will eliminate the need for LOAD_CONST 0 (None). Yes. > It's probably a wee bit faster and it makes the bytecode smaller, > because you don't need None in co_consts and you don't need an > argument to the bytecode. Yes. > Based on my cycle counter measurements before the conference, I > suspect the performance impact is, well, negligible. Regarding changing LOAD_CONST 0 to LOAD_NONE, yes. What's more interesting are the changes of LOAD_GLOBAL 'None' to one of LOAD_CONST 0 or LOAD_NONE. That could be changed to use LOAD_CONST 0 *now*, without adding a new bytecode, and we could get a better idea of how much performance it actually buys us in practice, since we get rid of two dict lookups (globals & builtins). That doesn't address the deprecation cycle, but it would be nice to see what the change would buy us. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From niemeyer@conectiva.com Fri Apr 26 15:29:52 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Fri, 26 Apr 2002 11:29:52 -0300 Subject: [Python-Dev] Making None a keyword In-Reply-To: <15561.24516.417362.240364@slothrop.zope.com> References: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> <15560.50336.965410.772225@slothrop.zope.com> <200204260317.g3Q3HZQ32684@pcp742651pcs.reston01.va.comcast.net> <15560.52437.47576.481530@slothrop.zope.com> <15560.58038.737344.840187@grendel.zope.com> <15561.24516.417362.240364@slothrop.zope.com> Message-ID: <20020426112951.B9148@ibook.distro.conectiva> > I assume LOAD_NONE will eliminate the need for LOAD_CONST 0 (None). > Instead of this: [...] > It's probably a wee bit faster and it makes the bytecode smaller, > because you don't need None in co_consts and you don't need an > argument to the bytecode. > > Based on my cycle counter measurements before the conference, I > suspect the performance impact is, well, negligible. Now I missed the point here.. :-) You told a new opcode was needed to make it faster, but at the same time you said the performance impact is negligible. Could you please clarify? -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From ark@research.att.com Fri Apr 26 15:33:11 2002 From: ark@research.att.com (Andrew Koenig) Date: 26 Apr 2002 10:33:11 -0400 Subject: [Python-Dev] an oddball alternative name for enumerate() Message-ID: What does the function do? It attaches a serial number to each element of a sequence. So my first thought was that it should be named serialize(), but of course that sounds like it's related to parallel processing. But what about inventory()? My mental image is of someone going around with a bunch of numbered stickers, attaching one to every relevant object. -- Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark From niemeyer@conectiva.com Fri Apr 26 15:33:51 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Fri, 26 Apr 2002 11:33:51 -0300 Subject: [Python-Dev] Making None a keyword In-Reply-To: <15561.25348.496531.621413@grendel.zope.com> References: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> <15560.50336.965410.772225@slothrop.zope.com> <200204260317.g3Q3HZQ32684@pcp742651pcs.reston01.va.comcast.net> <15560.52437.47576.481530@slothrop.zope.com> <15560.58038.737344.840187@grendel.zope.com> <15561.24516.417362.240364@slothrop.zope.com> <15561.25348.496531.621413@grendel.zope.com> Message-ID: <20020426113351.C9148@ibook.distro.conectiva> Hi Fred! > Regarding changing LOAD_CONST 0 to LOAD_NONE, yes. What's more > interesting are the changes of LOAD_GLOBAL 'None' to one of LOAD_CONST > 0 or LOAD_NONE. That could be changed to use LOAD_CONST 0 *now*, > without adding a new bytecode, and we could get a better idea of how > much performance it actually buys us in practice, since we get rid of > two dict lookups (globals & builtins). It's clear now.. thanks! > That doesn't address the deprecation cycle, but it would be nice to > see what the change would buy us. I'm going to work on it, and come back with some results. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From fdrake@acm.org Fri Apr 26 15:45:09 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 26 Apr 2002 10:45:09 -0400 Subject: [Python-Dev] an oddball alternative name for enumerate() In-Reply-To: References: Message-ID: <15561.26613.679271.862440@grendel.zope.com> Andrew Koenig writes: > What does the function do? It attaches a serial number to each > element of a sequence. So my first thought was that it should > be named serialize(), but of course that sounds like it's related > to parallel processing. Or still worse, marshaling data to a stream. > But what about inventory()? My mental image is of someone going > around with a bunch of numbered stickers, attaching one to > every relevant object. Hmmm... sounds like this function, whatever it gets called, will require a cartoon in the manual. ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From jeremy@zope.com Fri Apr 26 15:51:13 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Fri, 26 Apr 2002 10:51:13 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: <20020426112951.B9148@ibook.distro.conectiva> References: <200204260308.g3Q38ca32176@pcp742651pcs.reston01.va.comcast.net> <15560.50336.965410.772225@slothrop.zope.com> <200204260317.g3Q3HZQ32684@pcp742651pcs.reston01.va.comcast.net> <15560.52437.47576.481530@slothrop.zope.com> <15560.58038.737344.840187@grendel.zope.com> <15561.24516.417362.240364@slothrop.zope.com> <20020426112951.B9148@ibook.distro.conectiva> Message-ID: <15561.26977.47689.273355@slothrop.zope.com> I meant two say things, and tried to be clever at the expense of being understood :-(. I think the cleanest way to handle None as a keyword is via a LOAD_NONE opcode. LOAD_NONE would replace two different mechanisms that the compiler currently uses -- LOAD_GLOBAL 'None' and LOAD_CONST 0 (None). While I think LOAD_NONE would result in a tidy state of affairs, and I think it would be marginally faster, I don't think the performance effect would be measurable on any interesting benchmark. I don't care how much faster it is, though. Jeremy From Oleg Broytmann Fri Apr 26 16:29:49 2002 From: Oleg Broytmann (Oleg Broytmann) Date: Fri, 26 Apr 2002 19:29:49 +0400 Subject: [Python-Dev] Re: email, rfc 2231 In-Reply-To: <15561.19272.259210.135700@anthem.wooz.org>; from barry@zope.com on Fri, Apr 26, 2002 at 08:42:48AM -0400 References: <20020426114422.A24431@phd.pp.ru> <15561.19272.259210.135700@anthem.wooz.org> Message-ID: <20020426192949.C28624@phd.pp.ru> On Fri, Apr 26, 2002 at 08:42:48AM -0400, Barry A. Warsaw wrote: > A patch against Python CVS would be best. https://sourceforge.net/tracker/index.php?func=detail&aid=549133&group_id=5470&atid=305470 Oleg. -- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From tim.one@comcast.net Fri Apr 26 16:31:39 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 26 Apr 2002 11:31:39 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: <200204261242.g3QCguQ02366@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > ... > It needs to add 'None' as an alternative to all other places where > currently NAME occurs. Fudge. None makes no sense anywhere in, e.g., dotted_name: NAME ('.' NAME)* or global_stmt: 'global' NAME (',' NAME)* but I suppose it has to be allowed for in 2.3 anyway else an existing import None.None.None as None would yield a SyntaxError instead of a warning. Note too that we can't change the generated code at all for 2.3 unless under the control of a new future statement, else existing None-abusing code could break. From guido@python.org Fri Apr 26 16:34:10 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 11:34:10 -0400 Subject: [Python-Dev] re: PEP 279 revisited, formally In-Reply-To: Your message of "Wed, 24 Apr 2002 10:37:57 PDT." <3CC6ED75.7030208@ActiveState.com> References: <00b301c1eb88$225a56f0$56296c42@damien> <200204241249.g3OCnRn23155@pcp742651pcs.reston01.va.comcast.net> <3CC6DF9D.38254BF9@prescod.net> <3CC6ED75.7030208@ActiveState.com> Message-ID: <200204261534.g3QFYAh03096@pcp742651pcs.reston01.va.comcast.net> Thanks to all for the feedback. I think I've seen slightly more votes for enumerate than for itemize, and after some thought I think the LaTeX example (where enumerate creates numbered items) *does* help -- even if few Python users know LaTeX, I trust the linguistic sensibilities of LaTeX' author. So enumerate() it is. (Specifically not enum() because of the C/C++ meaning of that word.) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Apr 26 16:44:23 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 11:44:23 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: Your message of "Fri, 26 Apr 2002 11:31:39 EDT." References: Message-ID: <200204261544.g3QFiNH03283@pcp742651pcs.reston01.va.comcast.net> > [Guido] > > ... > > It needs to add 'None' as an alternative to all other places where > > currently NAME occurs. > > Fudge. None makes no sense anywhere in, e.g., > > dotted_name: NAME ('.' NAME)* > > or > > global_stmt: 'global' NAME (',' NAME)* > > but I suppose it has to be allowed for in 2.3 anyway else an existing Yes, that's (alas) the point. > import None.None.None as None > > would yield a SyntaxError instead of a warning. More realistically, def foo(a, None=None): if a is None: ... I found 4 examples of this in Zope3 alone, and MAL has some too. > Note too that we can't change the generated code at all for 2.3 > unless under the control of a new future statement, else existing > None-abusing code could break. A future statement makes no sense for this, so I guess all we can do in 2.3 is warn about use of None in all contexts except in expressions. This is really too bad -- I wish we could decide to just break code (since it's a clean break) but apparently that's unacceptable to the users. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Fri Apr 26 16:54:12 2002 From: skip@pobox.com (Skip Montanaro) Date: Fri, 26 Apr 2002 10:54:12 -0500 Subject: [Python-Dev] unicodeobject.c,2.139,2.140 checkin In-Reply-To: <3CC9104B.FC713BD1@lemburg.com> References: <1D0866D4-5895-11D6-A56C-003065517236@oratrix.com> <3CC9104B.FC713BD1@lemburg.com> Message-ID: <15561.30756.643852.63467@beluga.mojam.com> mal> In the long run, I think that the Unicode type should be used for mal> all text data and strings for binary data of unknown encoding. In that case, I would suggest you call "string" something else. Skip From pobrien@orbtech.com Fri Apr 26 17:00:16 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Fri, 26 Apr 2002 11:00:16 -0500 Subject: [Python-Dev] Making None a keyword In-Reply-To: Message-ID: [Tim Peters] > > Fudge. None makes no sense anywhere in, e.g., > > dotted_name: NAME ('.' NAME)* I know nothing about the compiler code, but what about something like: >>> None.__class__.__base__ >>> Or does this not apply because None is at the front? --- Patrick K. O'Brien Orbtech From tim.one@comcast.net Fri Apr 26 16:58:06 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 26 Apr 2002 11:58:06 -0400 Subject: [Python-Dev] re: PEP 279 revisited, formally In-Reply-To: <200204261534.g3QFYAh03096@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > ... > So enumerate() it is. (Specifically not enum() because of the C/C++ > meaning of that word.) The C/C++ meaning isn't a barrier to me: a C enum decl without embedded '=' must associate 0 with the first name, 1 with the second name, and so on. Indeed, if the Python enum returned pairs in (value, index) order, dict(enum(['apple', 'pear', 'godzilla']) would create the dict {'apple': 0, 'pear': 1, 'godzilla': 2} which is about as close to the C enum {apple, pear, godzilla}; /* now apple==0, pear==1, godzilla==2 */ as you can get with a Python function. From skip@pobox.com Fri Apr 26 17:00:26 2002 From: skip@pobox.com (Skip Montanaro) Date: Fri, 26 Apr 2002 11:00:26 -0500 Subject: [Python-Dev] Unicode In-Reply-To: <200204261223.g3QCNpN02195@pcp742651pcs.reston01.va.comcast.net> References: <3CC84D5F.32BA9085@lemburg.com> <200204251859.g3PIxTe16275@odiug.zope.com> <3CC91480.DF8A46ED@lemburg.com> <200204261223.g3QCNpN02195@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15561.31130.674403.256414@beluga.mojam.com> Guido> Also, once 8-bit strings are used for binary data only, I wonder Guido> if they shouldn't be more like Java's byte arrays -- Guido> i.e. mutable. And they don't need a literal notation. That's Guido> another major language change. :-( How so? In theory, all the ways you write string constructors today would eventually map to Unicode objects. I'm thinking of string literals and constructor functions. That can be handled with the usual "warn for awhile" mechanism. The array module could be used to manipulate mutable arrays of 8-bit data. While permeating the Python innards with Unicode objects would be a major change I don't see any big syntactic changes - or is that not what worries you? Skip From skip@pobox.com Fri Apr 26 17:07:26 2002 From: skip@pobox.com (Skip Montanaro) Date: Fri, 26 Apr 2002 11:07:26 -0500 Subject: [Python-Dev] Making None a keyword In-Reply-To: References: Message-ID: <15561.31550.380233.973078@beluga.mojam.com> Patrick> ... what about something like: >>> None.__class__.__base__ Patrick> Or does this not apply because None is at the front? I suspect it does apply. I doubt it's used much, but what would happen to types.NoneType? If None is a keyword, how will you generate it, or will it disappear when that happens? Skip From tim.one@comcast.net Fri Apr 26 17:11:29 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 26 Apr 2002 12:11:29 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: <200204261544.g3QFiNH03283@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > ... > A future statement makes no sense for this, Do you really want to give up the chance to tell your son that from __future__ import None once meant something in Python? That would be a great hack, esp. because in 2.4 it would also require horrid new trickery to ensure that it continued to work despite that None *would* be a keyword then . > so I guess all we can do in 2.3 is warn about use of None in all > contexts except in expressions. In that case there's really no point to changing the grammar now either, right? This is a context-sensitive warning. In that case, I'd like us to start warning about (at least) "non-expression" uses of 'True' and 'False' too. That is, if we can't change the generated code yet anyway, we may as well start warning about everything we *intend* to make keywords someday. The same context-senstive checks are needed for each such name. > This is really too bad -- I wish we could decide to just break code > (since it's a clean break) but apparently that's unacceptable to the > users. Delay 2.3 by 2.3 years and call it "Python 3" <0.9 wink>. From sholden@holdenweb.com Fri Apr 26 17:08:32 2002 From: sholden@holdenweb.com (Steve Holden) Date: Fri, 26 Apr 2002 12:08:32 -0400 Subject: [Python-Dev] Fun with numbers References: <15560.34601.661364.356755@anthem.wooz.org> <200204260102.g3Q12r927686@pcp742651pcs.reston01.va.comcast.net> Message-ID: <003701c1ed3c$9f6dd300$6300000a@holdenweb.com> ----- Original Message ----- From: "Guido van Rossum" To: "Barry A. Warsaw" Cc: Sent: Thursday, April 25, 2002 9:02 PM Subject: Re: [Python-Dev] Fun with numbers > > Tim, Jeremy, and I are having so much fun... > [...] > > >>> 23000 .__class__ = bool > > Debug memory block at address p=0x814f544: > > 485823496 bytes originally requested > > the 4 pad bytes at p-4 are not all FORBIDDENBYTE (0xfb): > > at p-4: 0x7a *** OUCH > > at p-3: 0x61 *** OUCH > > at p-2: 0xc8 *** OUCH > > at p-1: 0x3c *** OUCH > > the 4 pad bytes at tail=0x250a094c are Segmentation fault > > Ouch. Fixed in CVS for this particular case, but I think you may > still be able provoke this with a built-in type that derives from > another built-in type without adding any new fields, if the base type > has a funky allocator that the derived type doesn't inherit. Also, > the fix means that if an extension defines a type that inherits from > int but doesn't override tp_alloc and tp_free, it inherits a tp_alloc > that's not matched to the tp_free. But why would anyone do that...? > Probably because they don't know what they're doing. Expect to see my bug report in SourceForge. > Inspired by this, I tried something else. > > >>> print True > True > >>> True.__class__ = int > >>> print True > 1 > >>> > > There are no ill side effects. Handy for bool-haters. :-) > Now that's neat. regards Steve -- home: http://www.holdenweb.com/ Python Web Programming: http://pydish.holdenweb.com/pwp/ From tim.one@comcast.net Fri Apr 26 17:14:16 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 26 Apr 2002 12:14:16 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: Message-ID: >> dotted_name: NAME ('.' NAME)* [Patrick K. O'Brien] > I know nothing about the compiler code, but what about something like: > > >>> None.__class__.__base__ > > >>> > > Or does this not apply because None is at the front? Yup, good point, 'None' does make sense as the first NAME in the dotted_name production. From niemeyer@conectiva.com Fri Apr 26 17:20:40 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Fri, 26 Apr 2002 13:20:40 -0300 Subject: [Python-Dev] Making None a keyword In-Reply-To: References: <200204261242.g3QCguQ02366@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020426132039.B10512@ibook.distro.conectiva> > Note too that we can't change the generated code at all for 2.3 unless under > the control of a new future statement, else existing None-abusing code could > break. I'm trying to eat the cake and have it too, by detecting if None is being abused and disabling the change of LOAD_GLOBAL to LOAD_CONST. If it works (and probably will), we may then choose to include warnings, keep it that way forever, or whatever. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From mclay@nist.gov Fri Apr 26 17:20:41 2002 From: mclay@nist.gov (Michael McLay) Date: Fri, 26 Apr 2002 12:20:41 -0400 Subject: [Python-Dev] an oddball alternative name for enumerate() In-Reply-To: References: Message-ID: <200204261620.g3QGKZvU025640@email.nist.gov> On Friday 26 April 2002 10:33 am, Andrew Koenig wrote: > What does the function do? It attaches a serial number to each > element of a sequence. So my first thought was that it should > be named serialize(), but of course that sounds like it's related > to parallel processing. > > But what about inventory()? My mental image is of someone going > around with a bunch of numbered stickers, attaching one to > every relevant object. Inventory is a creative alternative, but it brings to mind the tallying of occurances of each item. The process of adding a serial number to something is an indexing operation. A non-programmer, who's job requires very precise use of language, thought indexing was the most appropriate term for the task as described. Cataloging was suggested as a weak second, but that term generally applies to assigning numbers to physical items. Assigning unique identifiers to abstract objects, such as laws or chemical compounds, is an indexing process. Unfortunately the PEP has rejected all varations of the term index: All of the names involving 'index' clashed with usage in database languages where indexing implies a sorting operation rather than linear sequencing. This seems like a weak reason for rejecting the accurate use of the term index in Python. Only people who use databases will think the term index means to sort. Python is a general purpose language. Many Python users will not find the usage confusing and will find all of the alternative terms confusing. I searched the Python document, using Thomas Heller's nifty "Index for Python manuals". There are 8 uses of the term index and none of them have to do with a sorting operation of a database. http://starship.python.net/crew/theller/pyhelp.cgi?keyword=index I also searched for 'indexing'. There were 0 hits on that term. I would suggest 'indexing' is the term that most closely describes the action of the function. My apologies to the database community for any confusion this might cause them. (Never mind they misused the term in the first place. If they ment sort then they should have used the word sort.) If indexing is still rejected then I will continue to be +0 on using serialize. From tim.one@comcast.net Fri Apr 26 17:22:45 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 26 Apr 2002 12:22:45 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: <15561.31550.380233.973078@beluga.mojam.com> Message-ID: [Skip Montanaro] > I doubt it's used much, but what would happen to types.NoneType? > If None is a keyword, how will you generate it, or will it disappear > when that happens? I don't see the problem here. It's currently generated via NoneType = type(None) and don't see why that would need to change. Use of 'None" in "rvalue" contexts couldn't be outlawed (what Guido called "except in expressions", in the futile hope that it would be clear ). From guido@python.org Fri Apr 26 17:26:39 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 12:26:39 -0400 Subject: [Python-Dev] Unicode In-Reply-To: Your message of "Fri, 26 Apr 2002 11:00:26 CDT." <15561.31130.674403.256414@beluga.mojam.com> References: <3CC84D5F.32BA9085@lemburg.com> <200204251859.g3PIxTe16275@odiug.zope.com> <3CC91480.DF8A46ED@lemburg.com> <200204261223.g3QCNpN02195@pcp742651pcs.reston01.va.comcast.net> <15561.31130.674403.256414@beluga.mojam.com> Message-ID: <200204261626.g3QGQdI03824@pcp742651pcs.reston01.va.comcast.net> > How so? In theory, all the ways you write string constructors today > would eventually map to Unicode objects. I'm thinking of string > literals and constructor functions. That can be handled with the > usual "warn for awhile" mechanism. The array module could be used > to manipulate mutable arrays of 8-bit data. While permeating the > Python innards with Unicode objects would be a major change I don't > see any big syntactic changes - or is that not what worries you? No syntactic changes, no. But the way we do things would become significantly different. And think of binary I/O vs. textual I/O -- currently, file.read() returns a string. Code dealing with binary files will look significantly different, and old code won't work. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Apr 26 17:29:18 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 12:29:18 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: Your message of "Fri, 26 Apr 2002 11:07:26 CDT." <15561.31550.380233.973078@beluga.mojam.com> References: <15561.31550.380233.973078@beluga.mojam.com> Message-ID: <200204261629.g3QGTIw03847@pcp742651pcs.reston01.va.comcast.net> > Patrick> ... what about something like: > > >>> None.__class__.__base__ > > > Patrick> Or does this not apply because None is at the front? > > I suspect it does apply. > > I doubt it's used much, but what would happen to types.NoneType? If > None is a keyword, how will you generate it, or will it disappear > when that happens? Nothing would change. None would be a literal keyword, but its value in an expression would be the same as before, so None.__class__ would work. Compare "foo".__class__. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Apr 26 17:30:32 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 12:30:32 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: Your message of "Fri, 26 Apr 2002 12:11:29 EDT." References: Message-ID: <200204261630.g3QGUXO03869@pcp742651pcs.reston01.va.comcast.net> > In that case there's really no point to changing the grammar now either, > right? This is a context-sensitive warning. In that case, I'd like us to > start warning about (at least) "non-expression" uses of 'True' and 'False' > too. That is, if we can't change the generated code yet anyway, we may as > well start warning about everything we *intend* to make keywords someday. > The same context-senstive checks are needed for each such name. Good point. Same question. Any takers? --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@zope.com Fri Apr 26 17:23:34 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Fri, 26 Apr 2002 12:23:34 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: <20020426132039.B10512@ibook.distro.conectiva> References: <200204261242.g3QCguQ02366@pcp742651pcs.reston01.va.comcast.net> <20020426132039.B10512@ibook.distro.conectiva> Message-ID: <15561.32518.230974.504717@slothrop.zope.com> >>>>> "GN" == Gustavo Niemeyer writes: >> Note too that we can't change the generated code at all for 2.3 >> unless under the control of a new future statement, else existing >> None-abusing code could break. GN> I'm trying to eat the cake and have it too, by detecting if None GN> is being abused and disabling the change of LOAD_GLOBAL to GN> LOAD_CONST. If it works (and probably will), we may then choose GN> to include warnings, keep it that way forever, or whatever. This can't be detected statically, because a global could be introduced dynamically: foo.py: def f(): return None >>> import foo >>> foo.None = 12 >>> foo.f() 12 Jeremy From guido@python.org Fri Apr 26 17:31:26 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 12:31:26 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: Your message of "Fri, 26 Apr 2002 12:14:16 EDT." References: Message-ID: <200204261631.g3QGVQM03881@pcp742651pcs.reston01.va.comcast.net> > Yup, good point, 'None' does make sense as the first NAME in the > dotted_name production. No, dotted_name is only used in import statements. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Fri Apr 26 17:32:28 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 26 Apr 2002 12:32:28 -0400 Subject: [Python-Dev] More deprecations In-Reply-To: <15561.19416.407233.566080@anthem.wooz.org> Message-ID: [Barry A. Warsaw] > This is emerging as a common theme, and a good one to adhere to. I > wonder if we should formalize it (i.e. target the stdlib to Python 2.1 > where possible)? -1. Using new features in the CVS libraries and tools is the only way we have to evaluate their usefulness, effectiveness, buginess and x-platform brittleness in anything approaching widely deployed code. From mal@lemburg.com Fri Apr 26 17:48:07 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 26 Apr 2002 18:48:07 +0200 Subject: [Python-Dev] unicodeobject.c,2.139,2.140 checkin References: <1D0866D4-5895-11D6-A56C-003065517236@oratrix.com> <3CC9104B.FC713BD1@lemburg.com> <200204261219.g3QCJci02177@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CC984C7.49D3684A@lemburg.com> Guido van Rossum wrote: > > > In the long run, I think that the Unicode type should be > > used for all text data and strings for binary data of unknown > > encoding. > > I think you're underestimating the tenacity of ASCII and Latin-1. This has nothing to do with a particular encoding, it's about meta data: much like you can use numbers to represent e.g. money or date/time values, you always need to store this meta information somewhere and object types are the ideal carrier for such meta data. It's an option which everybody working in the application space should consider. If you start working this way now, it'll pay off a few years ahead regardeless of whether it is useful for you now or not. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From jeremy@zope.com Fri Apr 26 17:53:53 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Fri, 26 Apr 2002 12:53:53 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: <200204261631.g3QGVQM03881@pcp742651pcs.reston01.va.comcast.net> References: <200204261631.g3QGVQM03881@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15561.34337.299235.100551@slothrop.zope.com> >>>>> "GvR" == Guido van Rossum writes: >> Yup, good point, 'None' does make sense as the first NAME in the >> dotted_name production. GvR> No, dotted_name is only used in import statements. None is a valid package name. Jeremy From mal@lemburg.com Fri Apr 26 18:03:12 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 26 Apr 2002 19:03:12 +0200 Subject: [Python-Dev] unicodeobject.c,2.139,2.140 checkin References: <1D0866D4-5895-11D6-A56C-003065517236@oratrix.com> <3CC9104B.FC713BD1@lemburg.com> <15561.30756.643852.63467@beluga.mojam.com> Message-ID: <3CC98850.2FFA3538@lemburg.com> Skip Montanaro wrote: > > mal> In the long run, I think that the Unicode type should be used for > mal> all text data and strings for binary data of unknown encoding. > > In that case, I would suggest you call "string" something else. That would call for world revolution and again put Unicode in a bad light. I'd rather not ;-) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From pobrien@orbtech.com Fri Apr 26 18:11:12 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Fri, 26 Apr 2002 12:11:12 -0500 Subject: [Python-Dev] Making None a keyword In-Reply-To: <200204261629.g3QGTIw03847@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido van Rossum] > > Nothing would change. None would be a literal keyword, but its value > in an expression would be the same as before, so None.__class__ would > work. Compare "foo".__class__. >>> ''.__class__ >>> [].__class__ >>> {}.__class__ >>> True.__class__ >>> 2.__class__ File "", line 1 2.__class__ ^ SyntaxError: invalid syntax >>> 2.2.__class__ >>> 2L.__class__ >>> Looks like int is one of the few literals that doesn't allow this. But floats and longs do. Hmmm. Just FYI. ;-) --- Patrick K. O'Brien Orbtech From guido@python.org Fri Apr 26 18:15:19 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 13:15:19 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: Your message of "Fri, 26 Apr 2002 12:53:53 EDT." <15561.34337.299235.100551@slothrop.zope.com> References: <200204261631.g3QGVQM03881@pcp742651pcs.reston01.va.comcast.net> <15561.34337.299235.100551@slothrop.zope.com> Message-ID: <200204261715.g3QHFJo04044@pcp742651pcs.reston01.va.comcast.net> > None is a valid package name. But it won't be once None is a keyword, so we should warn about its use as package or module name in 2.3 (or else we can never promote it to keyword). --Guido van Rossum (home page: http://www.python.org/~guido/) From aleax@aleax.it Fri Apr 26 18:13:43 2002 From: aleax@aleax.it (Alex Martelli) Date: Fri, 26 Apr 2002 19:13:43 +0200 Subject: [Python-Dev] Making None a keyword In-Reply-To: References: Message-ID: On Friday 26 April 2002 07:11 pm, Patrick K. O'Brien wrote: ... > >>> 2.__class__ This of course is being parsed as the floating literal 2. followed by the identifier __class__ ... > Looks like int is one of the few literals that doesn't allow this. But > floats and longs do. Hmmm. Just FYI. ;-) There's no int literal in the above. Try putting a space after the digit 2 , so that there IS an int literal ... Alex From martin@v.loewis.de Fri Apr 26 18:15:40 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 26 Apr 2002 19:15:40 +0200 Subject: [Python-Dev] pymalloc and overallocation (unicodeobject.c,2.139,2.140 checkin) In-Reply-To: <3CC90F2E.8235A5EA@lemburg.com> References: <3CC90F2E.8235A5EA@lemburg.com> Message-ID: "M.-A. Lemburg" writes: > The standard reasoning behind using overallocation for memory > management is that typical modern malloc()s don't really allocate > the memory until it is used (you know this anyway...), That is not true. Each malloc implementation I know will always iterate through the freelist to find an appriately-sized chunk, and go to the OS if it doesn't find one. Now, the *OS* might implement such allocations as a no-op, but on most hardware, it can do so only in units of memory pages (e.g. 4k). Most strings are smaller than a page, so if you allocate lots of strings, every page allocated from the system will be used as well (atleast to fill in the string header). With overallocation, you will indeed overallocate real pages, and consume real memory. > This makes overallocation ideal for the case where you don't know > the exact size in advance but where you can estimate a reasonable > upper bound. No. The overallocation has a real cost in terms of memory consumption. > As Martin's benchmark showed, the counting strategy is > faster for small chunks and this is probably due to the > fact that pymalloc manages these. I doubt this claim. > Since pymalloc cannot know that an algorithm wants to > use overallocation as memory allocation strategy, it > would probably help to find a way to tell pymalloc > about this fact. It could then either redirect the > request to the system malloc() or use a different > malloc strategy for these chunks. That won't help. Regards, Martin From mal@lemburg.com Fri Apr 26 18:16:38 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 26 Apr 2002 19:16:38 +0200 Subject: [Python-Dev] More deprecations References: <3CC823FC.CF687436@metaslash.com> <15560.48088.698840.716694@grendel.zope.com> <3CC8C38D.30009F83@metaslash.com> <3CC90752.82AE82A4@lemburg.com> <15561.19416.407233.566080@anthem.wooz.org> <200204261255.g3QCtZe02428@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CC98B76.8B0F9F44@lemburg.com> "Barry A. Warsaw" wrote: > > >>>>> "MAL" == M writes: > > MAL> In general, I think it's a good idea to be very careful about > MAL> introduce flashy new stuff to the std lib since it makes > MAL> backporting fixes harder, e.g. using Python 2.1 features is > MAL> OK, but not 2.2 features (if not absolutely necessary or only > MAL> if they are easily separatable from the rest of the code). > > This is emerging as a common theme, and a good one to adhere to. I > wonder if we should formalize it (i.e. target the stdlib to Python 2.1 > where possible)? Guido van Rossum wrote: > > What would the point be? How many people are moving stdlib modules to > previous releases? I think very few. The point is that in order to not make backporting patches for bug fix releases harder than necessariy, the code should not change in ways which would cause the backport to have to create a variant of the bug fix. This would introduce a branch and most probably new bugs. > I just added True/False to a whole bunch of places, and I'm sure with > each new enticing feature it will be harder to keep the stdlib 2.1 > compatible. True and False are not hard to backport, but any new features which are not replicable in Python 2.1 should not go into the stdlib until we drop support for it. Then, if we do, Python 2.2 should become the new reference and so on. > I'm all for marking certain modules as "have to be backwards > compatible" though -- e.g. sre. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From martin@v.loewis.de Fri Apr 26 18:16:32 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 26 Apr 2002 19:16:32 +0200 Subject: [Python-Dev] unicodeobject.c,2.139,2.140 checkin In-Reply-To: <200204261219.g3QCJci02177@pcp742651pcs.reston01.va.comcast.net> References: <1D0866D4-5895-11D6-A56C-003065517236@oratrix.com> <3CC9104B.FC713BD1@lemburg.com> <200204261219.g3QCJci02177@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > > In the long run, I think that the Unicode type should be > > used for all text data and strings for binary data of unknown > > encoding. > > I think you're underestimating the tenacity of ASCII and Latin-1. I agree with MAL: the run will just be *very* long. Regards, Martin From tim.one@comcast.net Fri Apr 26 18:27:37 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 26 Apr 2002 13:27:37 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: <15561.34337.299235.100551@slothrop.zope.com> Message-ID: [Tim] > Yup, good point, 'None' does make sense as the first NAME in the > dotted_name production. [Guido] > No, dotted_name is only used in import statements. [Jeremy] > None is a valid package name. But it won't be when None becomes a keyword. We shouldn't change the grammar now anyway, so let's fight instead about which names *should* become keywords eventually (== which should generate warnings in 2.3 for non-rvalue uses). import string import __builtin__ candidates = [k for k in dir(__builtin__) if k[0] in string.ascii_uppercase and not k.endswith('Error') and not k.endswith('Warning')] for k in candidates: print k displays: Ellipsis Exception False KeyboardInterrupt None NotImplemented StopIteration SystemExit True in current CVS. Ellipsis False None NotImplemented True are the builtin singleton data constants, so I'm +1 on those. The others are exception classes; -0 from me. From skip@pobox.com Fri Apr 26 18:39:19 2002 From: skip@pobox.com (Skip Montanaro) Date: Fri, 26 Apr 2002 12:39:19 -0500 Subject: [Python-Dev] Making None a keyword In-Reply-To: References: <15561.31550.380233.973078@beluga.mojam.com> Message-ID: <15561.37063.855246.215510@beluga.mojam.com> Tim> I don't see the problem here. It's currently generated via Tim> NoneType = type(None) duh... Sorry, my bad. Use of the term "keyword" was causing my sleep-deprived brain to think it couldn't be used in expressions. Obviously, that's not the case. All the current keywords in the language either identify statements statements (def, print, continue) or operators (or, and, in). Elevating "None" to keyword status would create a third type keyword. Maybe it should be referred to as a "reserved identifier" or something like that. Skip From martin@v.loewis.de Fri Apr 26 19:17:15 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 26 Apr 2002 20:17:15 +0200 Subject: [Python-Dev] Making None a keyword In-Reply-To: <15561.37063.855246.215510@beluga.mojam.com> References: <15561.31550.380233.973078@beluga.mojam.com> <15561.37063.855246.215510@beluga.mojam.com> Message-ID: Skip Montanaro writes: > Use of the term "keyword" was causing my sleep-deprived brain to think it > couldn't be used in expressions. Obviously, that's not the case. All the > current keywords in the language either identify statements statements (def, > print, continue) or operators (or, and, in). Elevating "None" to keyword > status would create a third type keyword. Maybe it should be referred to as > a "reserved identifier" or something like that. I'd put it into the class "literals": it is indeed a literal identifying the singleton NoneType instance. Of course, it will be the only literal that consists only of letters (it is *not* the only literal starting with a letter, nor the only literal ending with one). Regards, Martin From sholden@holdenweb.com Fri Apr 26 19:13:14 2002 From: sholden@holdenweb.com (Steve Holden) Date: Fri, 26 Apr 2002 14:13:14 -0400 Subject: [Python-Dev] Making None a keyword References: <200204261544.g3QFiNH03283@pcp742651pcs.reston01.va.comcast.net> Message-ID: <009701c1ed4e$09e55120$6300000a@holdenweb.com> ----- Original Message ----- From: "Guido van Rossum" To: "Tim Peters" Cc: Sent: Friday, April 26, 2002 11:44 AM Subject: Re: [Python-Dev] Making None a keyword > > [Guido] > > > ... > > > It needs to add 'None' as an alternative to all other places where > > > currently NAME occurs. > > > > Fudge. None makes no sense anywhere in, e.g., > > > > dotted_name: NAME ('.' NAME)* > > > > or > > > > global_stmt: 'global' NAME (',' NAME)* > > > > but I suppose it has to be allowed for in 2.3 anyway else an existing > > Yes, that's (alas) the point. > > > import None.None.None as None > > > > would yield a SyntaxError instead of a warning. > > More realistically, > > def foo(a, None=None): > if a is None: ... > > I found 4 examples of this in Zope3 alone, and MAL has some too. > > > Note too that we can't change the generated code at all for 2.3 > > unless under the control of a new future statement, else existing > > None-abusing code could break. > > A future statement makes no sense for this, so I guess all we can do > in 2.3 is warn about use of None in all contexts except in > expressions. > > This is really too bad -- I wish we could decide to just break code > (since it's a clean break) but apparently that's unacceptable to the > users. > Wouldn't it be nice if there were a collection of programs (perhaps those owned by registered paranoids, but whatever) that you could use to get statistics about breakage under some syntax change? Clearly extending this through to run-time would add a huge testing infrastructure that would need maintaining, but allowing people to add their own codes to the syntax-checker base might mollify them a bit about the prospect of future language change. Call it the PythOnGuard (TM) database. :-) - if your programs would break you'll be mailed before it's committed to production. Back to the yardwork ... regards Steve -- home: http://www.holdenweb.com/ Python Web Programming: http://pydish.holdenweb.com/pwp/ home: http://www.holdenweb.com/ Python Web Programming: http://pydish.holdenweb.com/pwp/ From sholden@holdenweb.com Fri Apr 26 19:14:35 2002 From: sholden@holdenweb.com (Steve Holden) Date: Fri, 26 Apr 2002 14:14:35 -0400 Subject: [Python-Dev] re: PEP 279 revisited, formally References: Message-ID: <009f01c1ed4e$3974ea40$6300000a@holdenweb.com> ----- Original Message ----- From: "Tim Peters" To: Sent: Friday, April 26, 2002 11:58 AM Subject: RE: [Python-Dev] re: PEP 279 revisited, formally > [Guido] > > ... > > So enumerate() it is. (Specifically not enum() because of the C/C++ > > meaning of that word.) > > The C/C++ meaning isn't a barrier to me: a C enum decl without embedded '=' > must associate 0 with the first name, 1 with the second name, and so on. > Indeed, if the Python enum returned pairs in (value, index) order, > > dict(enum(['apple', 'pear', 'godzilla']) > > would create the dict > > {'apple': 0, 'pear': 1, 'godzilla': 2} > > which is about as close to the C > > enum {apple, pear, godzilla}; > /* now apple==0, pear==1, godzilla==2 */ > > as you can get with a Python function. > +1 -- home: http://www.holdenweb.com/ Python Web Programming: http://pydish.holdenweb.com/pwp/ From tim.one@comcast.net Fri Apr 26 19:21:29 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 26 Apr 2002 14:21:29 -0400 Subject: [Python-Dev] pymalloc and overallocation (unicodeobject.c,2.139,2.140 checkin) In-Reply-To: Message-ID: [MAL] > The standard reasoning behind using overallocation for memory > management is that typical modern malloc()s don't really allocate > the memory until it is used (you know this anyway...), [Martin] > ... > Most strings are smaller than a page, so if you allocate lots of > strings, every page allocated from the system will be used as well > (at least to fill in the string header). With overallocation, you will > indeed overallocate real pages, and consume real memory. But Marc-Andre uses realloc at the end to return the excess. The excess bytes will get reused (and some returned yet again) by the next overallocation, and so on. The only memory *touched* by him is the only memory he actually needs in the end. Indeed, if strings are always smaller than a page, the aggregate overallocation at any point in a single-threaded program will be at worst a few pages total (overallocation is never more than a factor of 4, and the excess is always given back untouched). >> As Martin's benchmark showed, the counting strategy is >> faster for small chunks and this is probably due to the >> fact that pymalloc manages these. > I doubt this claim. If you both run the test with and without pymalloc enabled, the results should speak for themselves. I have not, but suspect the difference has more to do with that caches like small memory areas better than large ones, and especially when you crawl over a memory area twice. MAL, you should keep in mind that pymalloc is also managing the small chunks in your scheme: when you're fiddling with a 40-character Unicode string, an overallocation "by a factor of 4" only amounts to an 80-character UTF8 string. pymalloc handles blocks that small under either scheme, and indeed your current scheme is getting a speed benefit from that pymalloc currently refuses to copy the data to a smaller block when there's a shrinking realloc at the end. >> Since pymalloc cannot know that an algorithm wants to >> use overallocation as memory allocation strategy, it >> would probably help to find a way to tell pymalloc >> about this fact. It could then either redirect the >> request to the system malloc() or use a different >> malloc strategy for these chunks. Possibly, yes. It's still in need of quantifying the speed versus memory tradeoffs. pymalloc's current small-block realloc strategy favors speed, and going to the system malloc instead would be slower. You haven't yet told me that's what you actually want . From guido@python.org Fri Apr 26 19:26:35 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 14:26:35 -0400 Subject: [Python-Dev] More deprecations In-Reply-To: Your message of "Fri, 26 Apr 2002 19:16:38 +0200." <3CC98B76.8B0F9F44@lemburg.com> References: <3CC823FC.CF687436@metaslash.com> <15560.48088.698840.716694@grendel.zope.com> <3CC8C38D.30009F83@metaslash.com> <3CC90752.82AE82A4@lemburg.com> <15561.19416.407233.566080@anthem.wooz.org> <200204261255.g3QCtZe02428@pcp742651pcs.reston01.va.comcast.net> <3CC98B76.8B0F9F44@lemburg.com> Message-ID: <200204261826.g3QIQZm04171@pcp742651pcs.reston01.va.comcast.net> > True and False are not hard to backport, but any new > features which are not replicable in Python 2.1 should not > go into the stdlib until we drop support for it. Then, if we > do, Python 2.2 should become the new reference and so on. I think that puts the priorities backwards. If we can't develop the stdlib beyond what's supported by 2.1 we might as well stop developing. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Fri Apr 26 19:28:41 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 26 Apr 2002 14:28:41 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: <15561.37063.855246.215510@beluga.mojam.com> Message-ID: [Skip Montanaro] > ... > All the current keywords in the language either identify statements > statements (def, print, continue) or operators (or, and, in). > Elevating "None" to keyword status would create a third type keyword. > Maybe it should be referred to as a "reserved identifier" or something > like that. +1 from me. From guido@python.org Fri Apr 26 19:34:32 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 14:34:32 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: Your message of "Fri, 26 Apr 2002 13:27:37 EDT." References: Message-ID: <200204261834.g3QIYWp04258@pcp742651pcs.reston01.va.comcast.net> > Ellipsis > False > None > NotImplemented > True > > are the builtin singleton data constants, so I'm +1 on those. I'll go along with True and False, since I expect that these will be frequently used (though still not as frequently as None). Ellipsis is very rarely used (only by numeric code implementing multi-dimensional arrays), NotImplemented is used somewhat more but still pretty esoteric. So I'm -1 on those. Otherwise, you might as well start making all built-in functions keywords -- that would probably buy you more, too. E.g. if range were a keyword, the compiler could implement "for i in range(n)" without materializing the whole list at once. But I'm not proposing that; I think there's a better way to get the same effect without making everything a keyword. I just think that None is important enough to become a keyword. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Apr 26 19:40:44 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 14:40:44 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: Your message of "Fri, 26 Apr 2002 14:13:14 EDT." <009701c1ed4e$09e55120$6300000a@holdenweb.com> References: <200204261544.g3QFiNH03283@pcp742651pcs.reston01.va.comcast.net> <009701c1ed4e$09e55120$6300000a@holdenweb.com> Message-ID: <200204261840.g3QIeit04354@pcp742651pcs.reston01.va.comcast.net> > Wouldn't it be nice if there were a collection of programs (perhaps > those owned by registered paranoids, but whatever) that you could > use to get statistics about breakage under some syntax change? Yes. :-) But I think syntax changes aren't even the tip of the iceberg -- semantic changes are relevant too. > Clearly extending this through to run-time would add a huge testing > infrastructure that would need maintaining, but allowing people to > add their own codes to the syntax-checker base might mollify them a > bit about the prospect of future language change. > > Call it the PythOnGuard (TM) database. :-) - if your programs would > break you'll be mailed before it's committed to production. Sounds like some kind of huge test suite. The problem is, there's always *something* that breaks... :-( --Guido van Rossum (home page: http://www.python.org/~guido/) From martin@v.loewis.de Fri Apr 26 19:57:39 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 26 Apr 2002 20:57:39 +0200 Subject: [Python-Dev] pymalloc and overallocation (unicodeobject.c,2.139,2.140 checkin) In-Reply-To: References: Message-ID: Tim Peters writes: > But Marc-Andre uses realloc at the end to return the excess. The excess > bytes will get reused (and some returned yet again) by the next > overallocation, and so on. Right. I confused this with the fact that PyMem_Realloc won't return the excess memory, so the extra bytes in a small string will be wasted for the life time of the string object - that still could cause significant memory wastage. > MAL, you should keep in mind that pymalloc is also managing the small chunks > in your scheme: when you're fiddling with a 40-character Unicode string, an > overallocation "by a factor of 4" only amounts to an 80-character UTF8 > string. [I guess this is a terminology, not a math problem: a 40 character Unicode string has already 80 bytes; the UTF-8 of it can have up to 160 bytes]. Regards, Martin From mal@lemburg.com Fri Apr 26 20:02:58 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 26 Apr 2002 21:02:58 +0200 Subject: [Python-Dev] More deprecations References: <3CC823FC.CF687436@metaslash.com> <15560.48088.698840.716694@grendel.zope.com> <3CC8C38D.30009F83@metaslash.com> <3CC90752.82AE82A4@lemburg.com> <15561.19416.407233.566080@anthem.wooz.org> <200204261255.g3QCtZe02428@pcp742651pcs.reston01.va.comcast.net> <3CC98B76.8B0F9F44@lemburg.com> <200204261826.g3QIQZm04171@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3CC9A462.88687C94@lemburg.com> Guido van Rossum wrote: > > > True and False are not hard to backport, but any new > > features which are not replicable in Python 2.1 should not > > go into the stdlib until we drop support for it. Then, if we > > do, Python 2.2 should become the new reference and so on. > > I think that puts the priorities backwards. If we can't develop the > stdlib beyond what's supported by 2.1 we might as well stop > developing. Huh ? It's not as if Python 2.1 doesn't offer you any means to implement whatever is needed in the standard lib. Most things don't require using bleeding edge language features which give 2.1/2.2 bug fix champions a hard time. Just to clarify: I'm talking about Python 2.1 language features here. I'm not requesting to stop working on the standard lib, if that's what you're reading. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From guido@python.org Fri Apr 26 20:12:40 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 15:12:40 -0400 Subject: [Python-Dev] More deprecations In-Reply-To: Your message of "Fri, 26 Apr 2002 21:02:58 +0200." <3CC9A462.88687C94@lemburg.com> References: <3CC823FC.CF687436@metaslash.com> <15560.48088.698840.716694@grendel.zope.com> <3CC8C38D.30009F83@metaslash.com> <3CC90752.82AE82A4@lemburg.com> <15561.19416.407233.566080@anthem.wooz.org> <200204261255.g3QCtZe02428@pcp742651pcs.reston01.va.comcast.net> <3CC98B76.8B0F9F44@lemburg.com> <200204261826.g3QIQZm04171@pcp742651pcs.reston01.va.comcast.net> <3CC9A462.88687C94@lemburg.com> Message-ID: <200204261912.g3QJCen04912@pcp742651pcs.reston01.va.comcast.net> [MAL] > > > True and False are not hard to backport, but any new > > > features which are not replicable in Python 2.1 should not > > > go into the stdlib until we drop support for it. Then, if we > > > do, Python 2.2 should become the new reference and so on. [GvR] > > I think that puts the priorities backwards. If we can't develop the > > stdlib beyond what's supported by 2.1 we might as well stop > > developing. [MAL] > Huh ? It's not as if Python 2.1 doesn't offer you any > means to implement whatever is needed in the standard > lib. Most things don't require using bleeding edge language > features which give 2.1/2.2 bug fix champions a hard time. > > Just to clarify: > I'm talking about Python 2.1 language features here. > I'm not requesting to stop working on the standard lib, > if that's what you're reading. I understand you perfectly, and I disagree. If some code in a stdlib module can be written clearer or more efficiently by using a feature in 2.2 or 2.3, I don't see why we shouldn't use that feature, *unless* it's a module that is specifically marked to be exempt (like SRE). I won't start adding iterators and generators, nested scopes, new-style classes and so on just to make maintenance harder, but when writing new code or when revising existing code, we should use the features that make sense. Those features were all added because they improve the language's usability. Requiring the entire stdlib to be backwards compatible with 2.1 would be too difficult to do (and it's too late already). --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Fri Apr 26 20:25:54 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 26 Apr 2002 15:25:54 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: <200204261834.g3QIYWp04258@pcp742651pcs.reston01.va.comcast.net> Message-ID: >> Ellipsis >> False >> None >> NotImplemented >> True >> >> are the builtin singleton data constants, so I'm +1 on those. [Guido] > I'll go along with True and False, since I expect that these will be > frequently used (though still not as frequently as None). In a dynamic sense I bet they'll be used more often than None, in part because I'll steer people toward it . For years I've been steering people to manage their "dicts used as sets" via: for element in some_candidate_producer: dict_used_as_set[element] = 1 instead of using "None", for two reasons: first, I view the dict as mapping elements to "true", as in "yes, I'm present". The introduction of bool makes this very natural, as in ispresent = dict_used_as_set.get(element, False) The guts of loops like that have a high dynamic count. Second, using "1" instead of "None" is significantly faster today, and that can be important in short scripts doing data crunching (i.e., the dict setup can account for the bulk of such a program's runtime, and the faster string-keyed dict guts get, the more significant the fixed overhead of doing the "= None/1/True" part). Objective statistics : A module I've been writing for my own fun is up to 329 lines now. It contains "None" 8 times, and "True" and "False" 7 times each. The Nones are mostly used as argument defaults (so evaluated once per program run), and in one-time-per-call "was an argument specified?" tests at function starts. One None is used in an inner loop, in an assert that a certain value is not None. The dynamic count of True and False evaluations is going to swamp the dynamic count of None evaluations in this program. > Ellipsis is very rarely used (only by numeric code implementing > multi-dimensional arrays), NotImplemented is used somewhat more but > still pretty esoteric. So I'm -1 on those. Fine by me. So is this consensus? Start warning for non-rvalue uses of None False True and that's it for 2.3. From guido@python.org Fri Apr 26 20:46:36 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 15:46:36 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: Your message of "Fri, 26 Apr 2002 15:25:54 EDT." References: Message-ID: <200204261946.g3QJka705201@pcp742651pcs.reston01.va.comcast.net> > Fine by me. So is this consensus? Start warning for non-rvalue uses of > > None > False > True > > and that's it for 2.3. Yes. I don't need no steenkin' consensus. :-) Jeremy has started a patch: http://python.org/sf/549213 --Guido van Rossum (home page: http://www.python.org/~guido/) From sholden@holdenweb.com Fri Apr 26 20:50:10 2002 From: sholden@holdenweb.com (Steve Holden) Date: Fri, 26 Apr 2002 15:50:10 -0400 Subject: [Python-Dev] Making None a keyword References: <200204261544.g3QFiNH03283@pcp742651pcs.reston01.va.comcast.net> <009701c1ed4e$09e55120$6300000a@holdenweb.com> <200204261840.g3QIeit04354@pcp742651pcs.reston01.va.comcast.net> Message-ID: <00f401c1ed5b$93c80290$6300000a@holdenweb.com> [Me:] > > Wouldn't it be nice if there were a collection of programs (perhaps > > those owned by registered paranoids, but whatever) that you could > > use to get statistics about breakage under some syntax change? [Guido:] > > Yes. :-) > > But I think syntax changes aren't even the tip of the iceberg -- > semantic changes are relevant too. > Sorry, this is the syntax department. I think you need to speak to Mr. Tester in room 104. > > Clearly extending this through to run-time would add a huge testing > > infrastructure that would need maintaining, but allowing people to > > add their own codes to the syntax-checker base might mollify them a > > bit about the prospect of future language change. > > > > Call it the PythOnGuard (TM) database. :-) - if your programs would > > break you'll be mailed before it's committed to production. > > Sounds like some kind of huge test suite. The problem is, there's > always *something* that breaks... :-( > OK. The hugeness of the test suite was precisely what made me stick to syntax. Do you think quality would benefit by an enlargement of the test suite to non-distributed code? Couple it with a few polling buttons from "I wouldn't mind fixing this breakage" to "I'ma get my gun and come looking for you" and you might obtain a measure of resistance *from people whose code would actually be broken*. Clearly this should not necessarily be the arbiter of development, but it might allow you to tell people whose code hadn't broken so far to just PythOnGuard it and not complain until something *did* break. Also good for news releases: "What breaks in this version?" (I can see Paul Rubin loving that one). regards Steve -- home: http://www.holdenweb.com/ Python Web Programming: http://pydish.holdenweb.com/pwp/ From guido@python.org Fri Apr 26 21:08:09 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 16:08:09 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: Your message of "Fri, 26 Apr 2002 15:50:10 EDT." <00f401c1ed5b$93c80290$6300000a@holdenweb.com> References: <200204261544.g3QFiNH03283@pcp742651pcs.reston01.va.comcast.net> <009701c1ed4e$09e55120$6300000a@holdenweb.com> <200204261840.g3QIeit04354@pcp742651pcs.reston01.va.comcast.net> <00f401c1ed5b$93c80290$6300000a@holdenweb.com> Message-ID: <200204262008.g3QK8Av05272@pcp742651pcs.reston01.va.comcast.net> > Sorry, this is the syntax department. I think you need to speak to Mr. > Tester in room 104. I think I'll continue arguing in my spare time. :) > > > Clearly extending this through to run-time would add a huge testing > > > infrastructure that would need maintaining, but allowing people to > > > add their own codes to the syntax-checker base might mollify them a > > > bit about the prospect of future language change. > > > > > > Call it the PythOnGuard (TM) database. :-) - if your programs would > > > break you'll be mailed before it's committed to production. > > > > Sounds like some kind of huge test suite. The problem is, there's > > always *something* that breaks... :-( > > > OK. The hugeness of the test suite was precisely what made me stick > to syntax. Do you think quality would benefit by an enlargement of > the test suite to non-distributed code? The quality of what? We need lots more test, probably 40% of the C code isn't covered by the test suite. > Couple it with a few polling buttons from "I wouldn't mind fixing > this breakage" to "I'ma get my gun and come looking for you" and you > might obtain a measure of resistance *from people whose code would > actually be broken*. Clearly this should not necessarily be the > arbiter of development, but it might allow you to tell people whose > code hadn't broken so far to just PythOnGuard it and not complain > until something *did* break. I think it's too simplistic to expect much out of testing a large body of real life code. Much of the expected breakage only shows up when an application is run on real-life data. Maybe running PyChecker on a large body of code would be a good idea, except that then the code would first have to be made PyChecker-clean -- and that's a moving target. > Also good for news releases: "What breaks in this version?" (I can > see Paul Rubin loving that one). It's easy to list the things that could potentially break. It's usually impossible to find whether they are used in a particular body of code without running a thorough test suite *for that code*. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Fri Apr 26 21:59:23 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 26 Apr 2002 16:59:23 -0400 Subject: [Python-Dev] pymalloc and overallocation (unicodeobject.c,2.139,2.140 checkin) In-Reply-To: Message-ID: [Tim] >> But Marc-Andre uses realloc at the end to return the excess. The >> excess bytes will get reused (and some returned yet again) by the >> next overallocation, and so on. [Martin] > Right. I confused this with the fact that PyMem_Realloc won't return > the excess memory, PyMem_Realloc does whatever the system realloc does -- PyMem_Realloc doesn't go thru pymalloc today (except in a PYMALLOC_DEBUG build). Doesn't matter, though, since strings use the PyObject_{Malloc, Free, Realloc} family today, and that does use pymalloc. OTOH, there's no reason PyObject_Realloc *has* to hang on to all small-block memory on a shrinking realloc, and there's no reason pymalloc couldn't grow another realloc entry point specifying what the caller wants a shrinking realloc to do. These things are all easy to change, but I don't know what's truly desirable. Note another subtlety: I expect you brought up PyMem_Realloc because unicodeobject.c uses the PyMem_XYZ family for managing the PyUnicodeObject.str member today. That means it normally never uses pymalloc at all, except to allocate fixed-size PyUnicodeObject structs (which use the PyObject_XYZ memory family). I don't know whether that's the best idea, but that's how it is today. pymalloc gets into this because PyUnicode_EncodeUTF8 returns a plain string object, and the latter uses pymalloc today. > so the extra bytes in a small string will be wasted for the life > time of the string object - that still could cause significant memory > wastage. It could. Python generally aims to optimize the expected case, not jump thru hoops to avoid worst cases (else we wouldn't use dicts at all ). But I don't know what the expected case is here, and given how often I use Unicode in my own work it could be I'll never have a clue. Note that the expected uses of Unicode strings makes no difference to PyUnicode_EncodeUTF8: what counts there is the expected lifetimes and sizes of the "plain" utf8-encoded PyStringObjects it computes. Indeed, pymalloc has almost no implications for Unicode beyond the encode-as-a-plain-string functions (unless unicodeobject.c is changed to manage the PyUnicodeObject.str member using pymalloc too, as plain strings do today). >> MAL, you should keep in mind that pymalloc is also managing the >> small chunks in your scheme: when you're fiddling with a 40-character >> Unicode string, an overallocation "by a factor of 4" only amounts to >> an 80-character UTF8 string. > [I guess this is a terminology, not a math problem: Nope! Turns out it was an hallucination problem . > a 40 character Unicode string has already 80 bytes; the UTF-8 of > it can have up to 160 bytes]. You're right, of course. The conclusion doesn't change, though: that's still in the range of block pymalloc handles (and will remain so unless I reduce pymalloc's small-object threshold below what's needed for pymalloc to handle small dicts on its own -- which I'm unlikely to do). From guido@python.org Fri Apr 26 22:09:38 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 17:09:38 -0400 Subject: [Python-Dev] pymalloc and overallocation (unicodeobject.c,2.139,2.140 checkin) In-Reply-To: Your message of "Fri, 26 Apr 2002 16:59:23 EDT." References: Message-ID: <200204262109.g3QL9dk05619@pcp742651pcs.reston01.va.comcast.net> [Tim] > The conclusion doesn't change, though: that's still in the range of > block pymalloc handles (and will remain so unless I reduce > pymalloc's small-object threshold below what's needed for pymalloc > to handle small dicts on its own -- which I'm unlikely to do). Would it make sense to change the Unicode object to use pymalloc, and to change the UTF-8 codec to count the bytes if the shortest possible output would fit in a pymalloc block? (I guess this means that the length of the Unicode string should be less than SMALL_REQUEST_THRESHOLD - currently 256.) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Apr 26 22:26:59 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 17:26:59 -0400 Subject: [Python-Dev] Use for enumerate() In-Reply-To: Your message of "Fri, 26 Apr 2002 11:34:10 EDT." Message-ID: <200204262126.g3QLQxs05666@pcp742651pcs.reston01.va.comcast.net> Here's a cute use for enumerate(), now that it's checked in. I was reading the section on files in the Python Cookbook manuscript. It recommends using linecache() if you need to get a particular line from a file. I agree that this is a good approach if you may need multiple lines from the same file, but if you know that all you need is one line from one file, linecache does too much (it reads the whole file, and keeps all of it in memory). A function that reads forward until the right line is of course easily written for any version of Python; but enumerate() seems a particularly good fit here since it avoids the need to have a separate counter. This also shows how useful it is that files are iterators! def getline(filename, lineno): if lineno < 1: return '' lineno -= 1 f = open(filename) for i, line in enumerate(f): if i == lineno: break else: line = '' f.close() return line Challenge 1: do it faster. Challenge 2: do it with less code. Challenge 3: do it faster and with less code. Rules: the semantics must be the same: always close the file, return '' for lineno out of range; lineno < 1 should not open the file. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Fri Apr 26 22:47:54 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 26 Apr 2002 17:47:54 -0400 Subject: [Python-Dev] pymalloc and overallocation (unicodeobject.c,2.139,2.140 checkin) In-Reply-To: <200204262109.g3QL9dk05619@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > Would it make sense to change the Unicode object to use pymalloc, and > to change the UTF-8 codec to count the bytes if the shortest possible > output would fit in a pymalloc block? These are independent questions, and I don't know how to answer either unless you give me a test program that prints the value of the function you're trying to minimize <0.7 wink>. The Unicode object currenly uses quite an elaborate free list, caching both PyUnicodeObject structs (which currently use pymalloc), and their str members (which currently do not). Whether the str member uses pymalloc really doesn't have anything to do with what the UTF8 encoder function does (it returns plain strings, and those already use pymalloc today -- and it's not entirely clear whether they should either!). Counting the bytes in the UTF8 decoder could work well, independent of that: if the result is known to fit in a pymalloc block, just do it; as soon as it's known that it won't, overallocate with assurance that the system realloc will give back everything that isn't used. In the latter case I believe the code could be made much simpler, by doing a factor-of-4 overallocation from the start (it currently tries 2, then 3, then 4, with a bunch of embedded-in-the-loops tests to prevent overwrites; I'm not sure why it bothers with this staggered scheme, since it's going to touch exactly as much memory as it actually needs regardless, and give all the rest back untouched). > (I guess this means that the length of the Unicode string should be > less than SMALL_REQUEST_THRESHOLD - currently 256.) For a start, yes. I'd stick a "Py_" in front of that symbol and expose it then. The cutoff test would also have to take into account the size of the result's PyStringObject header (the whole stringobject enchilada counts against the threshold). From niemeyer@conectiva.com Fri Apr 26 22:53:48 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Fri, 26 Apr 2002 18:53:48 -0300 Subject: [Python-Dev] Making None a keyword In-Reply-To: <200204261946.g3QJka705201@pcp742651pcs.reston01.va.comcast.net> References: <200204261946.g3QJka705201@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020426185348.A11547@ibook.distro.conectiva> > Yes. I don't need no steenkin' consensus. :-) > > Jeremy has started a patch: http://python.org/sf/549213 Too bad.. it was mostly working. :-) Anything else in the top of your mind!? -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From mgilfix@eecs.tufts.edu Fri Apr 26 23:05:54 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Fri, 26 Apr 2002 18:05:54 -0400 Subject: [Python-Dev] timeoutsocket patch Message-ID: <20020426180554.J18749@eecs.tufts.edu> I'm working with Bernard Yue to implement a timeout socket patch. Our question: is it fair to assume that any OS that supports sockets also supports select? Or do have to have added checks? -- Mike -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From martin@v.loewis.de Fri Apr 26 23:14:34 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 27 Apr 2002 00:14:34 +0200 Subject: [Python-Dev] pymalloc and overallocation (unicodeobject.c,2.139,2.140 checkin) In-Reply-To: <200204262109.g3QL9dk05619@pcp742651pcs.reston01.va.comcast.net> References: <200204262109.g3QL9dk05619@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > Would it make sense to change the Unicode object to use pymalloc, and > to change the UTF-8 codec to count the bytes if the shortest possible > output would fit in a pymalloc block? (I guess this means that the > length of the Unicode string should be less than > SMALL_REQUEST_THRESHOLD - currently 256.) Given my measurements, that would make sense. I suspect that counting small strings is quite efficient, so that the overhead of iterating over the string twice hides in the noise of additional invocations. Regards, Martin From martin@v.loewis.de Fri Apr 26 23:08:03 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 27 Apr 2002 00:08:03 +0200 Subject: [Python-Dev] pymalloc and overallocation (unicodeobject.c,2.139,2.140 checkin) In-Reply-To: References: Message-ID: Tim Peters writes: > > Right. I confused this with the fact that PyMem_Realloc won't return > > the excess memory, > > PyMem_Realloc does whatever the system realloc does -- PyMem_Realloc doesn't > go thru pymalloc today (except in a PYMALLOC_DEBUG build). Doesn't matter, > though, since strings use the PyObject_{Malloc, Free, Realloc} family today, > and that does use pymalloc. That's what I mean (I'm *really* confused about memory family APIs, ever since everything changed :-) > OTOH, there's no reason PyObject_Realloc *has* to hang on to all > small-block memory on a shrinking realloc, and there's no reason > pymalloc couldn't grow another realloc entry point specifying what > the caller wants a shrinking realloc to do. These things are all > easy to change, but I don't know what's truly desirable. Neither do I. To establish whether releasing the extra memory is worth the effort would depend on knowledge whether the object will be long-living; neither pymalloc nor its caller is able to tell. > Note another subtlety: I expect you brought up PyMem_Realloc because > unicodeobject.c uses the PyMem_XYZ family for managing the > PyUnicodeObject.str member today. No, because I assumed PyMem_Realloc was a synonym for PyObject_Realloc. > That means it normally never uses pymalloc at all, except to > allocate fixed-size PyUnicodeObject structs (which use the > PyObject_XYZ memory family). I don't know whether that's the best > idea, but that's how it is today. I do think that the Unicode data should be managed by pymalloc as well. Of course, DecodeUTF8 would then raise the same issue: decoding UTF-8 doesn't know how many characters you'll get, either. This currently does not try to be clever, but allocates enough memory for the worst case. Regards, Martin From guido@python.org Fri Apr 26 23:46:57 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 26 Apr 2002 18:46:57 -0400 Subject: [Python-Dev] pymalloc and overallocation (unicodeobject.c,2.139,2.140 checkin) In-Reply-To: Your message of "27 Apr 2002 00:08:03 +0200." References: Message-ID: <200204262246.g3QMkvk05868@pcp742651pcs.reston01.va.comcast.net> Yet another idea: decode into a fixed-size stack-allocated buffer. If it fits in that buffer, use PyString_FromStringAndSize(). Otherwise, do the overallocation thing. --Guido van Rossum (home page: http://www.python.org/~guido/) From mgilfix@eecs.tufts.edu Fri Apr 26 23:46:23 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Fri, 26 Apr 2002 18:46:23 -0400 Subject: [Python-Dev] timeoutsocket patch In-Reply-To: <20020426180554.J18749@eecs.tufts.edu>; from mgilfix@eecs.tufts.edu on Fri, Apr 26, 2002 at 06:05:54PM -0400 References: <20020426180554.J18749@eecs.tufts.edu> Message-ID: <20020426184623.M18749@eecs.tufts.edu> Oh yeah, and could a few brave souls volunteer to do some testing on MS and MAC? The patch is complete at this point but needs some testin'. -- Mike On Fri, Apr 26 @ 18:05, Michael Gilfix wrote: > I'm working with Bernard Yue to implement a timeout socket > patch. Our question: is it fair to assume that any OS that > supports sockets also supports select? Or do have to have > added checks? > > -- Mike > > -- > Michael Gilfix > mgilfix@eecs.tufts.edu > > For my gpg public key: > http://www.eecs.tufts.edu/~mgilfix/contact.html > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev `-> (mgilfix) -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From pobrien@orbtech.com Sat Apr 27 00:31:29 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Fri, 26 Apr 2002 18:31:29 -0500 Subject: [Python-Dev] Use for enumerate() In-Reply-To: <200204262126.g3QLQxs05666@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido van Rossum] > > def getline(filename, lineno): > if lineno < 1: > return '' > lineno -= 1 > f = open(filename) > for i, line in enumerate(f): > if i == lineno: > break > else: > line = '' > f.close() > return line > > Challenge 1: do it faster. > > Challenge 2: do it with less code. > > Challenge 3: do it faster and with less code. > > Rules: the semantics must be the same: always close the file, return > '' for lineno out of range; lineno < 1 should not open the file. I doubt it's faster and it's barely less code (I had to resist the urge to add a try/finally), but here is the only decent variation I could come up with: def getline(filename, lineno): line = '' if lineno > 0: lineno -= 1 f = open(filename) for i, s in enumerate(f): if i == lineno: line = s break f.close() return line I'm ready to give up now. What's the fastest, smallest solution? --- Patrick K. O'Brien Orbtech From DavidA@ActiveState.com Sat Apr 27 01:06:28 2002 From: DavidA@ActiveState.com (David Ascher) Date: Fri, 26 Apr 2002 17:06:28 -0700 Subject: [Python-Dev] Use for enumerate() References: Message-ID: <3CC9EB84.8050203@ActiveState.com> def getline(filename, lineno): if lineno >= 1: for i, line in enumerate(open(filename)): if i == lineno - 1: return line return '' Untested (I don't have a build quite recent enough). This is for Challenge 2. The lineno-1 makes it likely slower than the others. If there was an optional argument to enumerate specifying 'startat=1', then I might catch up. From pobrien@orbtech.com Sat Apr 27 01:19:58 2002 From: pobrien@orbtech.com (Patrick K. O'Brien) Date: Fri, 26 Apr 2002 19:19:58 -0500 Subject: [Python-Dev] Use for enumerate() In-Reply-To: <3CC9EB84.8050203@ActiveState.com> Message-ID: [David Ascher] > > def getline(filename, lineno): > if lineno >= 1: > for i, line in enumerate(open(filename)): > if i == lineno - 1: > return line > return '' But you broke the "always close the file" rule. Cheater!!! --- Patrick K. O'Brien Orbtech From DavidA@ActiveState.com Sat Apr 27 01:24:12 2002 From: DavidA@ActiveState.com (David Ascher) Date: Fri, 26 Apr 2002 17:24:12 -0700 Subject: [Python-Dev] Use for enumerate() References: Message-ID: <3CC9EFAC.4050209@ActiveState.com> Patrick K. O'Brien wrote: >[David Ascher] > >>def getline(filename, lineno): >> if lineno >= 1: >> for i, line in enumerate(open(filename)): >> if i == lineno - 1: >> return line >> return '' >> > >But you broke the "always close the file" rule. Cheater!!! > The file will get closed. I'm just not guaranteeing when =). From pyth@devel.trillke.net Sat Apr 27 01:47:19 2002 From: pyth@devel.trillke.net (holger krekel) Date: Sat, 27 Apr 2002 02:47:19 +0200 Subject: [Python-Dev] Use for enumerate() In-Reply-To: <200204262126.g3QLQxs05666@pcp742651pcs.reston01.va.comcast.net> References: <200204262126.g3QLQxs05666@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020427004719.GE22555@devel.trillke> On Fri, Apr 26, 2002 at 05:26:59PM -0400, Guido van Rossum wrote: > Challenge 2: do it with less code. and more functionality (you can pass -1,-2...): def getline(filename, lineno): l=[''] if lineno: f=open(filename) l+=f.read().split('\n') f.close() return l[len(l)>abs(lineno) and lineno] regards, holger From tim.one@comcast.net Sat Apr 27 01:56:31 2002 From: tim.one@comcast.net (Tim Peters) Date: Fri, 26 Apr 2002 20:56:31 -0400 Subject: [Python-Dev] Use for enumerate() In-Reply-To: <200204262126.g3QLQxs05666@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > Here's a cute use for enumerate(), now that it's checked in. > ... > A function that reads forward until the right line is of course easily > written for any version of Python; but enumerate() seems a > particularly good fit here since it avoids the need to have a separate > counter. This also shows how useful it is that files are iterators! > > def getline(filename, lineno): > if lineno < 1: > return '' > lineno -= 1 > f = open(filename) > for i, line in enumerate(f): > if i == lineno: > break > else: > line = '' > f.close() > return line > > Challenge 1: do it faster. Challenge 0: Stop posting Python code with stinking tabs . The attached g2 is about 60% quicker across the text files I tried it on, using this timing driver (which sums the time across all interesting inputs; note that, overall, this is quadratic-time in the number or lines for either method): FNAME = "whatever" def drive(): from time import clock as now f = file(FNAME) n = len(f.readlines()) f.close() indices = range(n+2) for func in getline, g2: start = now() for i in indices: func(FNAME, i) finish = now() print func.__name__, finish - start > Challenge 2: do it with less code. Measured by bytes, lines, statements, or conceptual pain ? > Challenge 3: do it faster and with less code. That's a tough one. Here's g2. It's a non-ambitious member of the "reduce the number of trips around the eval loop" school of optimization: def g2(filename, lineno): offset = lineno - 1 if offset < 0: return '' line = '' f = file(filename) lines = True while lines: lines = f.readlines(5000) n = len(lines) if offset >= n: offset -= n continue line = lines[offset] break f.close() return line From pyth@devel.trillke.net Sat Apr 27 03:41:23 2002 From: pyth@devel.trillke.net (holger krekel) Date: Sat, 27 Apr 2002 04:41:23 +0200 Subject: [Python-Dev] Use for enumerate() In-Reply-To: References: <200204262126.g3QLQxs05666@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020427024123.GF22555@devel.trillke> On Fri, Apr 26, 2002 at 08:56:31PM -0400, Tim Peters wrote: > The attached g2 is about 60% quicker across the text files I tried it on, > using this timing driver (which sums the time across all interesting inputs; > note that, overall, this is quadratic-time in the number or lines for either > method): measured with your driver script the following code is about 10-20% faster than 'g2' and it gets close to > > Challenge 3: do it faster and with less code. def g3(filename, lineno): if lineno>0: f = file(filename) while lineno>0: read = f.read(1024) count = read.count('\n') lineno-=count or lineno f.close() if lineno Message-ID: On Fri, 26 Apr 2002, Guido van Rossum wrote: > def getline(filename, lineno): > if lineno < 1: > return '' > lineno -= 1 > f = open(filename) > for i, line in enumerate(f): > if i == lineno: > break > else: > line = '' > f.close() > return line > > > Challenge 2: do it with less code. def getline(filename, lineno): if lineno < 1: return '' f = open(filename) for line in f: lineno -= 1 if not lineno: break else: line = '' f.close() return line -- Greg Ball From wolfson@midway.uchicago.edu Sat Apr 27 04:21:55 2002 From: wolfson@midway.uchicago.edu (Ben Wolfson) Date: Fri, 26 Apr 2002 22:21:55 -0500 (CDT) Subject: [Python-Dev] Use for enumerate Message-ID: (I'm not subscribed so this won't thread right, but oh well) This doesn't preserve the same semantics as the original, in that it always reads the entire file, but it is awfully short: def getline(filename, lineno): return (lineno < 1 and ('',) or ([ln for (i,ln) in enumerate(open(filename)) if i==lineno-1] or ('',)))[0] -- BTR YOU HEAR ME, BEN WOLFSON?!? YOUR SUPERCILIOUS CONTEMPT POWERS CANNOT WITHSTAND THE POWER OF MY BOORISH *DIMWIT* *RAY*!!!! -- revjack From wolfson@midway.uchicago.edu Sat Apr 27 04:27:12 2002 From: wolfson@midway.uchicago.edu (Ben Wolfson) Date: Fri, 26 Apr 2002 22:27:12 -0500 (CDT) Subject: [Python-Dev] Re: Use for enumerate In-Reply-To: Message-ID: On Fri, 26 Apr 2002, Ben Wolfson wrote: >(I'm not subscribed so this won't thread right, but oh well) > >This doesn't preserve the same semantics as the original, in that it >always reads the entire file, but it is awfully short: That is, it always reads it if it opens it at all. It preserves the semantics of not opening the file if lineno < 1. -- BTR YOU HEAR ME, BEN WOLFSON?!? YOUR SUPERCILIOUS CONTEMPT POWERS CANNOT WITHSTAND THE POWER OF MY BOORISH *DIMWIT* *RAY*!!!! -- revjack From tim.one@comcast.net Sat Apr 27 05:24:46 2002 From: tim.one@comcast.net (Tim Peters) Date: Sat, 27 Apr 2002 00:24:46 -0400 Subject: [Python-Dev] Use for enumerate() In-Reply-To: <20020427024123.GF22555@devel.trillke> Message-ID: [holger krekel] > measured with your driver script the following code > is about 10-20% faster than 'g2' and it gets close to > > > > Challenge 3: do it faster and with less code. > > def g3(filename, lineno): > if lineno>0: > f = file(filename) > while lineno>0: > read = f.read(1024) > count = read.count('\n') > lineno-=count or lineno > f.close() > if lineno return read.split('\n')[lineno+count-1] > return '' > > vertically it's one line less but it stretches > a bit horizontally. But it is more portable :-) What if (the simplest example of what can go wrong) the first line is more than 1024 characters long, and the caller asks for lineno 1? The argument to f.readlines(5000) is just a hint, and Python does whatever is necessary under the covers to ensure only complete lines are returned (even if it has to read millions of bytes to get a full line). The argument to f.read(1024) is a maximum, whether or not it finds a newline. Of course Guido never explicitly said it had to return a correct answer, in which case I vote for g=lambda*a:'' as both shortest and fastest by any measures . From tim.one@comcast.net Sat Apr 27 05:42:20 2002 From: tim.one@comcast.net (Tim Peters) Date: Sat, 27 Apr 2002 00:42:20 -0400 Subject: [Python-Dev] pymalloc and overallocation (unicodeobject.c,2.139,2.140 checkin) In-Reply-To: <200204262246.g3QMkvk05868@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > Yet another idea: decode into a fixed-size stack-allocated buffer. If > it fits in that buffer, use PyString_FromStringAndSize(). Otherwise, > do the overallocation thing. How come we can't use a version of C that pulls this trick for us automatically ? We do end up there a lot. Here's a patch that does it for PyUnicode_EncodeUTF8: http://www.python.org/sf/549375 From tim.one@comcast.net Sat Apr 27 06:01:03 2002 From: tim.one@comcast.net (Tim Peters) Date: Sat, 27 Apr 2002 01:01:03 -0400 Subject: [Python-Dev] pymalloc and overallocation (unicodeobject.c,2.139,2.140 checkin) In-Reply-To: Message-ID: [martin@v.loewis.de] > That's what I mean (I'm *really* confused about memory family APIs, > ever since everything changed :-) Here's the in-depth course: PyMem_xyz calls the platform malloc/realloc/free (fiddled for x-platform uniformity in NULL and 0 handling) PyObject_xyz calls pymalloc's malloc/realloc/free and instead of a dozen layers of indirection we've now got crushingly straightforward WYSIWYG preprocessor blocks like: #ifdef WITH_PYMALLOC #ifdef PYMALLOC_DEBUG #define PyObject_MALLOC _PyObject_DebugMalloc #define PyObject_Malloc _PyObject_DebugMalloc #define PyObject_REALLOC _PyObject_DebugRealloc #define PyObject_Realloc _PyObject_DebugRealloc #define PyObject_FREE _PyObject_DebugFree #define PyObject_Free _PyObject_DebugFree #else /* WITH_PYMALLOC && ! PYMALLOC_DEBUG */ #define PyObject_MALLOC PyObject_Malloc #define PyObject_REALLOC PyObject_Realloc #define PyObject_FREE PyObject_Free #endif #else /* ! WITH_PYMALLOC */ #define PyObject_MALLOC PyMem_MALLOC #define PyObject_REALLOC PyMem_REALLOC #define PyObject_FREE PyMem_FREE #endif /* WITH_PYMALLOC */ #define PyObject_Del PyObject_Free #define PyObject_DEL PyObject_FREE /* for source compatibility with 2.2 */ #define _PyObject_Del PyObject_Free All the names you love are still there, it's just that most of them are redundant now . > ... > I do think that the Unicode data should be managed by pymalloc as > well. Well, that largely depends on how big these suckers are. Calling PyObject_XYZ adds real overhead if pymalloc can't handle the requested size: all the overhead of the system routines, + the overhead of pymalloc figuring out it can't handle it. I expect it's also not good to mix pymalloc with custom free lists: you hold on to one object from a pymalloc pool, and it prevents the entire pool from getting recycled for another size class. So if you want to investigate using pymalloc more heavily for Unicode objects, I suggest two things: 1. Get rid of the Unicode-specific free list. 2. Change the object layout to embed the str member storage, just as PyStringObject does. #1 is pretty localized, but #2 would require changing a lot of code. > Of course, DecodeUTF8 would then raise the same issue: decoding > UTF-8 doesn't know how many characters you'll get, either. This > currently does not try to be clever, but allocates enough memory for > the worst case. I just put a patch up on SourceForge that's *less* clever, but shouldn't waste any memory in the end. I expect you'll be happy with it, or rant inconsolably. It's all the same to me . From python@rcn.com Sat Apr 27 06:08:43 2002 From: python@rcn.com (Raymond Hettinger) Date: Sat, 27 Apr 2002 01:08:43 -0400 Subject: [Python-Dev] Use for enumerate() Message-ID: <000f01c1eda9$9ad08fa0$25d8accf@othello> > Challenge 3: do it faster and with less code. def getline(filename, lineno): if lineno < 1: return '' f = open(filename) i, line = zip(xrange(lineno), f)[-1] f.close() if i+1 == lineno: return line return '' To keep to the spirit of the challenge, I'm ignoring that the function is i/o bound which would lead to using an 'rb' read and doing .finds or .counts on '\n'. The approach is to vectorize, trading away memory allocation time and xrange time to save the overhead of the pure Python loop and test cycle. The test is saved by taking advantage of zip's feature which stops when the first iterator is exhausted. Raymond Hettinger From tim.one@comcast.net Sat Apr 27 06:56:56 2002 From: tim.one@comcast.net (Tim Peters) Date: Sat, 27 Apr 2002 01:56:56 -0400 Subject: [Python-Dev] Use for enumerate() In-Reply-To: <000f01c1eda9$9ad08fa0$25d8accf@othello> Message-ID: >> Challenge 3: do it faster and with less code. [Raymond Hettinger] > def getline(filename, lineno): > if lineno < 1: > return '' > f = open(filename) > i, line = zip(xrange(lineno), f)[-1] > f.close() > if i+1 == lineno: > return line > return '' Hmm. On my box it's a little slower than Guido's getline on my standard test, here calling that function g3 (g2 and the timing driver were posted before; the input is Zope's DateTime.py, a 1657-line Python source file): getline 4.85231314638 g2 2.8915829967 g3 5.19037613772 That's a curious result, since, as you say: > The approach is to vectorize, trading away memory > allocation time and xrange time to save the overhead > of the pure Python loop and test cycle. It gets a speed boost to below 5.0 if I use range instead of xrange. It suggests this alternative, which is a tiny bit shorter and significantly faster than Guido's: def g4(filename, lineno): if lineno < 1: return '' f = open(filename) get = iter(f).next try: for i in range(lineno): line = get() except StopIteration: pass f.close() return line That weighs in at 4.04 seconds on my test case. I think the lesson to take is that building gobs of 2-tuples is more expensive than taking the same number of quick trips around the eval loop. Guido's and your function both build gobs of 2-tuples, while the zippier g4 and much zippier g2 avoid that. > ... > The test is saved by taking advantage of zip's feature > which stops when the first iterator is exhausted. It is clever! Too bad it's pig slow . From mgilfix@eecs.tufts.edu Sat Apr 27 07:11:56 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Sat, 27 Apr 2002 02:11:56 -0400 Subject: [Python-Dev] Use for enumerate() In-Reply-To: ; from tim.one@comcast.net on Sat, Apr 27, 2002 at 12:24:46AM -0400 References: <20020427024123.GF22555@devel.trillke> Message-ID: <20020427021155.A28402@eecs.tufts.edu> On Sat, Apr 27 @ 00:24, Tim Peters wrote: > Of course Guido never explicitly said it had to return a correct answer, in > which case I vote for > > g=lambda*a:'' > > as both shortest and fastest by any measures . Just return before you start :) -- Mike -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From pyth@devel.trillke.net Sat Apr 27 11:23:00 2002 From: pyth@devel.trillke.net (holger krekel) Date: Sat, 27 Apr 2002 12:23:00 +0200 Subject: [Python-Dev] Use for enumerate() In-Reply-To: References: <20020427024123.GF22555@devel.trillke> Message-ID: <20020427102300.GG22555@devel.trillke> On Sat, Apr 27, 2002 at 12:24:46AM -0400, Tim Peters wrote: > [holger krekel] > > measured with your driver script the following code > > is about 10-20% faster than 'g2' and it gets close to > > > > > > Challenge 3: do it faster and with less code. > > > > def g3(filename, lineno): > > if lineno>0: > > f = file(filename) > > while lineno>0: > > read = f.read(1024) > > count = read.count('\n') > > lineno-=count or lineno > > f.close() > > if lineno > return read.split('\n')[lineno+count-1] > > return '' > > > > vertically it's one line less but it stretches > > a bit horizontally. But it is more portable :-) > > What if (the simplest example of what can go wrong) the first line is more > than 1024 characters long, and the caller asks for lineno 1? right. i noticed after sending it that you also need to change lineno-= count or lineno into lineno-= count or read and lineno and the problem of lines longer>1024 and around the 1024 boundary remains. Solving this probably blows the code by several lines. But anyway, isn't reading and counting still a faster technique than using readlines() esp. for larger files? holger From guido@python.org Sat Apr 27 14:26:17 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 27 Apr 2002 09:26:17 -0400 Subject: [Python-Dev] Use for enumerate() In-Reply-To: Your message of "Sat, 27 Apr 2002 01:08:43 EDT." <000f01c1eda9$9ad08fa0$25d8accf@othello> References: <000f01c1eda9$9ad08fa0$25d8accf@othello> Message-ID: <200204271326.g3RDQII06636@pcp742651pcs.reston01.va.comcast.net> > > Challenge 3: do it faster and with less code. > > def getline(filename, lineno): > if lineno < 1: > return '' > f = open(filename) > i, line = zip(xrange(lineno), f)[-1] > f.close() > if i+1 == lineno: > return line > return '' Cute, but it builds up a list containing all the lines up to lineno. An implicit part of the exercise (sorry for not making this explicit) was to avoid this -- IOW it should work even if the file is too large to fit in memory (as long as each individual line fits in memory). --Guido van Rossum (home page: http://www.python.org/~guido/) From python@rcn.com Sat Apr 27 17:52:08 2002 From: python@rcn.com (Raymond Hettinger) Date: Sat, 27 Apr 2002 12:52:08 -0400 Subject: [Python-Dev] Use for enumerate() References: <000f01c1eda9$9ad08fa0$25d8accf@othello> <200204271326.g3RDQII06636@pcp742651pcs.reston01.va.comcast.net> Message-ID: <000701c1ee0b$df191b20$1eea7ad1@othello> > > > Challenge 3: do it faster and with less code. > it should work even if the file is too large > to fit in memory (as long as each individual line fits in memory). def getline(filename, lineno): # Second attempt if lineno < 1: return '' f = open(filename) reduce(lambda x,y: f.readline(), xrange(lineno-1), None) return (f.readline(), f.close())[0] Arghh, must resist lambda. Must not use reduce. Must avoid tuple tricks. ... GvR's code is too powerful I can't resist Raymond Hettinger From martin@v.loewis.de Sat Apr 27 17:55:20 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 27 Apr 2002 18:55:20 +0200 Subject: [Python-Dev] pymalloc and overallocation (unicodeobject.c,2.139,2.140 checkin) In-Reply-To: References: Message-ID: Tim Peters writes: > I just put a patch up on SourceForge that's *less* clever, but shouldn't > waste any memory in the end. I expect you'll be happy with it, or rant > inconsolably. It's all the same to me . Not that you care, but I'm happy with it :-) Martin From paul@prescod.net Sat Apr 27 18:07:21 2002 From: paul@prescod.net (Paul Prescod) Date: Sat, 27 Apr 2002 10:07:21 -0700 Subject: [Python-Dev] CHUNKSIZE Message-ID: <3CCADAC9.CE349EEC@prescod.net> xreadlines' default CHUNKSIZE is 8K. minidom defaults to double that. Have those values been keeping up with machine RAM inflation and should they? I presume others have thought this through but my probably-naive impression is that getting data in such little bytes is a waste of time on machines with tens or hundreds of megabytes of RAM. Paul Prescod From mgilfix@eecs.tufts.edu Sat Apr 27 18:12:13 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Sat, 27 Apr 2002 13:12:13 -0400 Subject: [Python-Dev] CHUNKSIZE In-Reply-To: <3CCADAC9.CE349EEC@prescod.net>; from paul@prescod.net on Sat, Apr 27, 2002 at 10:07:21AM -0700 References: <3CCADAC9.CE349EEC@prescod.net> Message-ID: <20020427131213.C1588@eecs.tufts.edu> This comment is more of a question.. Does the chunksize not have to do with how the filesystem stores blocks? I *think* it's better to retrieve in blocks that best match the size of the filesystem. So 8K would make sense since many FS's have 4K chunks and many have 16K chunks, etc... I'm probably totally wrong so someone please tell me why :) On Sat, Apr 27 @ 10:07, Paul Prescod wrote: > xreadlines' default CHUNKSIZE is 8K. minidom defaults to double that. > > Have those values been keeping up with machine RAM inflation and should > they? I presume others have thought this through but my probably-naive > impression is that getting data in such little bytes is a waste of time > on machines with tens or hundreds of megabytes of RAM. -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From python@rcn.com Sat Apr 27 18:20:59 2002 From: python@rcn.com (Raymond Hettinger) Date: Sat, 27 Apr 2002 13:20:59 -0400 Subject: Fw: [Python-Dev] Use for enumerate() Message-ID: <000b01c1ee0f$e6f94320$1eea7ad1@othello> > > > > Challenge 3: do it faster and with less code. > > it should work even if the file is too large > > to fit in memory (as long as each individual line fits in memory). def getline(filename, lineno): if lineno < 1: return '' f = open(filename) g = f.readline return (reduce(lambda x,y: g(), range(lineno), ''), f.close())[0] This incorporates a couple of ideas inspired by Tim's post. Raymond Hettinger From ikjos@email.uophx.edu Sat Apr 27 18:50:39 2002 From: ikjos@email.uophx.edu (Ian Kjos) Date: Sat, 27 Apr 2002 12:50:39 -0500 Subject: [Python-Dev] Re: Use for enumerate References: <20020427160005.11973.89266.Mailman@mail.python.org> Message-ID: <001001c1ee14$0bb2ac20$0dce4542@austin.rr.com> Towards the "short code" challenge: def g(n, x): if x<= 0: return '' f=file(n) for i in range(x): l = f.readline() f.close() return l It's tested on 2.2.1, and correctly returns '' for out of range. It won't stop at EOF, though. PS: This is a good argument for pre-object-death destructors a'la C++. I think resources should be finalized as soon as they are no longer accessable through other means. I realize that there are issues to be resolved; perl does it by following the law of most (!) astonishment. /me dives into some documentation. ----- Original Message ----- From: To: Sent: Saturday, April 27, 2002 11:00 AM Subject: Python-Dev digest, Vol 1 #2187 - 5 msgs > Send Python-Dev mailing list submissions to > python-dev@python.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.python.org/mailman/listinfo/python-dev > or, via email, send a message with subject or body 'help' to > python-dev-request@python.org > > You can reach the person managing the list at > python-dev-admin@python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Python-Dev digest..." > > > Today's Topics: > > 1. Re: Use for enumerate() (Raymond Hettinger) > 2. RE: Use for enumerate() (Tim Peters) > 3. Re: Use for enumerate() (Michael Gilfix) > 4. Re: Use for enumerate() (holger krekel) > 5. Re: Use for enumerate() (Guido van Rossum) > > --__--__-- > > Message: 1 > From: "Raymond Hettinger" > To: > Subject: Re: [Python-Dev] Use for enumerate() > Date: Sat, 27 Apr 2002 01:08:43 -0400 > > > Challenge 3: do it faster and with less code. > > def getline(filename, lineno): > if lineno < 1: > return '' > f = open(filename) > i, line = zip(xrange(lineno), f)[-1] > f.close() > if i+1 == lineno: > return line > return '' > > To keep to the spirit of the challenge, I'm ignoring that > the function is i/o bound which would lead to using > an 'rb' read and doing .finds or .counts on '\n'. > > The approach is to vectorize, trading away memory > allocation time and xrange time to save the overhead > of the pure Python loop and test cycle. > > The test is saved by taking advantage of zip's feature > which stops when the first iterator is exhausted. > > > Raymond Hettinger > > > > > > --__--__-- > > Message: 2 > Date: Sat, 27 Apr 2002 01:56:56 -0400 > From: Tim Peters > Subject: RE: [Python-Dev] Use for enumerate() > To: python-dev@python.org > > >> Challenge 3: do it faster and with less code. > > [Raymond Hettinger] > > def getline(filename, lineno): > > if lineno < 1: > > return '' > > f = open(filename) > > i, line = zip(xrange(lineno), f)[-1] > > f.close() > > if i+1 == lineno: > > return line > > return '' > > Hmm. On my box it's a little slower than Guido's getline on my standard > test, here calling that function g3 (g2 and the timing driver were > posted before; the input is Zope's DateTime.py, a 1657-line Python source > file): > > getline 4.85231314638 > g2 2.8915829967 > g3 5.19037613772 > > That's a curious result, since, as you say: > > > The approach is to vectorize, trading away memory > > allocation time and xrange time to save the overhead > > of the pure Python loop and test cycle. > > It gets a speed boost to below 5.0 if I use range instead of xrange. > > It suggests this alternative, which is a tiny bit shorter and significantly > faster than Guido's: > > def g4(filename, lineno): > if lineno < 1: > return '' > f = open(filename) > get = iter(f).next > try: > for i in range(lineno): line = get() > except StopIteration: > pass > f.close() > return line > > That weighs in at 4.04 seconds on my test case. > > I think the lesson to take is that building gobs of 2-tuples is more > expensive than taking the same number of quick trips around the eval loop. > Guido's and your function both build gobs of 2-tuples, while the zippier g4 > and much zippier g2 avoid that. > > > ... > > The test is saved by taking advantage of zip's feature > > which stops when the first iterator is exhausted. > > It is clever! Too bad it's pig slow . > > > > > --__--__-- > > Message: 3 > Date: Sat, 27 Apr 2002 02:11:56 -0400 > From: Michael Gilfix > To: Tim Peters > Cc: python-dev@python.org > Subject: Re: [Python-Dev] Use for enumerate() > Reply-To: mgilfix@eecs.tufts.edu > > On Sat, Apr 27 @ 00:24, Tim Peters wrote: > > Of course Guido never explicitly said it had to return a correct answer, in > > which case I vote for > > > > g=lambda*a:'' > > > > as both shortest and fastest by any measures . > > Just return before you start :) > > -- Mike > > -- > Michael Gilfix > mgilfix@eecs.tufts.edu > > For my gpg public key: > http://www.eecs.tufts.edu/~mgilfix/contact.html > > > > --__--__-- > > Message: 4 > Date: Sat, 27 Apr 2002 12:23:00 +0200 > From: holger krekel > To: Tim Peters > Cc: python-dev@python.org > Subject: Re: [Python-Dev] Use for enumerate() > > On Sat, Apr 27, 2002 at 12:24:46AM -0400, Tim Peters wrote: > > [holger krekel] > > > measured with your driver script the following code > > > is about 10-20% faster than 'g2' and it gets close to > > > > > > > > Challenge 3: do it faster and with less code. > > > > > > def g3(filename, lineno): > > > if lineno>0: > > > f = file(filename) > > > while lineno>0: > > > read = f.read(1024) > > > count = read.count('\n') > > > lineno-=count or lineno > > > f.close() > > > if lineno > > return read.split('\n')[lineno+count-1] > > > return '' > > > > > > vertically it's one line less but it stretches > > > a bit horizontally. But it is more portable :-) > > > > What if (the simplest example of what can go wrong) the first line is more > > than 1024 characters long, and the caller asks for lineno 1? > > right. i noticed after sending it that you also need to > change > lineno-= count or lineno > into > lineno-= count or read and lineno > > and the problem of lines longer>1024 and around the 1024 boundary remains. > Solving this probably blows the code by several lines. > > But anyway, isn't reading and counting still a faster technique than > using readlines() esp. for larger files? > > holger > > > > --__--__-- > > Message: 5 > To: "Raymond Hettinger" > cc: python-dev@python.org > Subject: Re: [Python-Dev] Use for enumerate() > From: Guido van Rossum > Date: Sat, 27 Apr 2002 09:26:17 -0400 > > > > Challenge 3: do it faster and with less code. > > > > def getline(filename, lineno): > > if lineno < 1: > > return '' > > f = open(filename) > > i, line = zip(xrange(lineno), f)[-1] > > f.close() > > if i+1 == lineno: > > return line > > return '' > > Cute, but it builds up a list containing all the lines up to lineno. > An implicit part of the exercise (sorry for not making this explicit) > was to avoid this -- IOW it should work even if the file is too large > to fit in memory (as long as each individual line fits in memory). > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > > > > --__--__-- > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > > > End of Python-Dev Digest > From barry@zope.com Sat Apr 27 19:49:23 2002 From: barry@zope.com (Barry A. Warsaw) Date: Sat, 27 Apr 2002 14:49:23 -0400 Subject: [Python-Dev] Re: Use for enumerate References: <20020427160005.11973.89266.Mailman@mail.python.org> <001001c1ee14$0bb2ac20$0dce4542@austin.rr.com> Message-ID: <15562.62131.337533.344976@anthem.wooz.org> >>>>> "IK" == Ian Kjos writes: IK> PS: This is a good argument for pre-object-death destructors IK> a'la C++. I think resources should be finalized as soon as IK> they are no longer accessable through other means. I realize IK> that there are issues to be resolved; perl does it by IK> following the law of most (!) astonishment. /me dives into IK> some documentation. Here's a 5 minute toy example... -Barry -------------------- snip snip --------------------/tmp/play.py class resource: def __init__(self, resource, func): self.__resource = resource self.__func = func def __call__(self, *args, **kws): self.__resource.acquire() try: rtn = self.__func(*args, **kws) finally: self.__resource.release() return rtn class FileResource: def __init__(self, filename): self._filename = filename self._fp = None self.firstbyte = resource(self, self.firstbyte) def acquire(self): assert self._fp is None self._fp = file(self._filename) def release(self): assert self._fp is not None self._fp.close() self._fp = None def firstbyte(self): self._fp.seek(0) return self._fp.read(1) fr = FileResource('/tmp/play.py') print 'firstbyte:', fr.firstbyte() assert fr._fp is None From tim.one@comcast.net Sat Apr 27 19:56:42 2002 From: tim.one@comcast.net (Tim Peters) Date: Sat, 27 Apr 2002 14:56:42 -0400 Subject: [Python-Dev] _PyString_Resize Message-ID: Word to the wise: learn how _PyString_Resize works. I just fixed dozens of misues in the code base, and beefed up the docs. Here's the checkin comment: """ Repair widespread misuse of _PyString_Resize. Since it's clear people don't understand how this function works, also beefed up the docs. The most common usage error is of this form (often spread out across gotos): if (_PyString_Resize(&s, n) < 0) { Py_DECREF(s); s = NULL; goto outtahere; } The error is that if _PyString_Resize runs out of memory, it automatically decrefs the input string object s (which also deallocates it, since its refcount must be 1 upon entry), and sets s to NULL. So if the "if" branch ever triggers, it's an error to call Py_DECREF(s): s is already NULL! A correct way to write the above is the simpler: if (_PyString_Resize(&s, n) < 0) goto outtahere; """ Even better, *most* of the time you can just do _PyString_Resize(&s, n); return s; at the end of a function. This works correctly in all respects regardless of whether _PyString_Resize() succeeds or fails. Note that there's also no need to store a trailing \0 byte yourself before calling _PyString_Resize: it does that automatically (for all the PyString_ alloc and realloc functions, the size you pass in is one less than the number of bytes actually allocated for string space, and a trailing \0 byte is always appended by magic in successful cases). From magnus@hetland.org Sat Apr 27 20:02:40 2002 From: magnus@hetland.org (Magnus Lie Hetland) Date: Sat, 27 Apr 2002 21:02:40 +0200 Subject: [Python-Dev] re: PEP 279 revisited, formally In-Reply-To: ; from tim.one@comcast.net on Fri, Apr 26, 2002 at 11:58:06AM -0400 References: <200204261534.g3QFYAh03096@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020427210240.A16214@idi.ntnu.no> Tim Peters : > > [Guido] > > ... > > So enumerate() it is. (Specifically not enum() because of the C/C++ > > meaning of that word.) > > The C/C++ meaning isn't a barrier to me: a C enum decl without embedded '=' > must associate 0 with the first name, 1 with the second name, and so on. > Indeed, if the Python enum returned pairs in (value, index) order, > > dict(enum(['apple', 'pear', 'godzilla']) > > would create the dict > > {'apple': 0, 'pear': 1, 'godzilla': 2} > > which is about as close to the C > > enum {apple, pear, godzilla}; > /* now apple==0, pear==1, godzilla==2 */ > > as you can get with a Python function. +1 -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.org From smurf@noris.de Sat Apr 27 20:30:57 2002 From: smurf@noris.de (Matthias Urlichs) Date: Sat, 27 Apr 2002 21:30:57 +0200 Subject: [Python-Dev] Unicode entities in XML cause problems :-( Message-ID: <20020427213057.F30164@noris.de> Playing around with xml.dom.minidom, I noticed that this beast is perfectly able to read HTML which it can't print: >>> import xml.dom.minidom as md >>> d=md.parseString("bߐ")) >>> d.writexml(sys.stdout) ... UnicodeError: ASCII encoding error: ordinal not in range(128) Ouch. Scanning the sources, which revealed various ways to replace '&' with '&' but no generic codec for [ht|x]ml-escaped character entities. Thus, my proposal (which I'm going to implement since I need it...) is to write such a codec. For simplicity, I propose to accept ü and € and friends, but to emit them as Ӓ (or whatever). After this codec is written, all occurrences of string.replace('&','&') (and vice versa) within the standard library can be replaced with the appropriate encode/decode methods. Thoughts? Or am I totally blind, such a codec already exists, and I have missed it? -- Matthias Urlichs | noris network AG | http://smurf.noris.de/ From mgilfix@eecs.tufts.edu Sat Apr 27 20:47:58 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Sat, 27 Apr 2002 15:47:58 -0400 Subject: [Python-Dev] Unicode entities in XML cause problems :-( In-Reply-To: <20020427213057.F30164@noris.de>; from smurf@noris.de on Sat, Apr 27, 2002 at 09:30:57PM +0200 References: <20020427213057.F30164@noris.de> Message-ID: <20020427154758.G1588@eecs.tufts.edu> I came across this myself before I joined the list. My general rule was to always convert unicode to strings with something like: "%s" %unicode (I don't remember if I avoided str because it also returned unicode) for any internal use. I think the context was I wanted to use a type attribute from an xml tag to instantiate an object whose class I retrieved from a dict. So I had something like: module = self.record['module'] if not resources.__dict__.has_key (module): raise RuntimeError, "Attempted to retrieve data from non-existant resource module: %s" %module code = resources.__dict__[module] obj = apply (code, [ ], self.record['args']) ... where module was a unicode string. This was an example where unicode sorta transparently pissed me off because it behaved just like a string in so many ways but wasn't. -- Mike On Sat, Apr 27 @ 21:30, Matthias Urlichs wrote: > Playing around with xml.dom.minidom, I noticed that this beast is > perfectly able to read HTML which it can't print: > > >>> import xml.dom.minidom as md > >>> d=md.parseString("bߐ")) > >>> d.writexml(sys.stdout) > ... > UnicodeError: ASCII encoding error: ordinal not in range(128) > > Ouch. -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From paul@prescod.net Sat Apr 27 21:32:58 2002 From: paul@prescod.net (Paul Prescod) Date: Sat, 27 Apr 2002 13:32:58 -0700 Subject: [Python-Dev] Unicode entities in XML cause problems :-( References: <20020427213057.F30164@noris.de> Message-ID: <3CCB0AFA.E9651C0F@prescod.net> Matthias Urlichs wrote: > > Playing around with xml.dom.minidom, I noticed that this beast is > perfectly able to read HTML which it can't print: > > >>> import xml.dom.minidom as md > >>> d=md.parseString("bߐ")) > >>> d.writexml(sys.stdout) > ... > UnicodeError: ASCII encoding error: ordinal not in range(128) "sys.stdout" doesn't know what to do with Unicode. Wrap it in an encoder (usually UTF-8) using the codecs module. I agree that this is a usability problem but it isn't a bug and I think you've mischaracterized the source of the problem. Paul Prescod From paul@prescod.net Sat Apr 27 21:34:30 2002 From: paul@prescod.net (Paul Prescod) Date: Sat, 27 Apr 2002 13:34:30 -0700 Subject: [Python-Dev] CHUNKSIZE References: <3CCADAC9.CE349EEC@prescod.net> <20020427131213.C1588@eecs.tufts.edu> Message-ID: <3CCB0B56.B910D3E9@prescod.net> I think you're wrong because a) block sizes vary too much across file system types and OS types, b) the effects of hardware and OS caching and pre-fetching would greatly obscure the actual block size. Paul Prescod From mgilfix@eecs.tufts.edu Sat Apr 27 21:37:45 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Sat, 27 Apr 2002 16:37:45 -0400 Subject: [Python-Dev] CHUNKSIZE In-Reply-To: <3CCB0B56.B910D3E9@prescod.net>; from paul@prescod.net on Sat, Apr 27, 2002 at 01:34:30PM -0700 References: <3CCADAC9.CE349EEC@prescod.net> <20020427131213.C1588@eecs.tufts.edu> <3CCB0B56.B910D3E9@prescod.net> Message-ID: <20020427163745.N1588@eecs.tufts.edu> Yeah, you're probably right. Hopefully someone knowledgeable on the subject will enlighten us. It does beg the question though: Why isn't readline mmap'd? Would that be more efficient? -- Mike On Sat, Apr 27 @ 13:34, Paul Prescod wrote: > I think you're wrong because a) block sizes vary too much across file > system types and OS types, b) the effects of hardware and OS caching and > pre-fetching would greatly obscure the actual block size. > > Paul Prescod > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev `-> (paul) -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From guido@python.org Sat Apr 27 22:59:51 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 27 Apr 2002 17:59:51 -0400 Subject: [Python-Dev] CHUNKSIZE In-Reply-To: Your message of "Sat, 27 Apr 2002 16:37:45 EDT." <20020427163745.N1588@eecs.tufts.edu> References: <3CCADAC9.CE349EEC@prescod.net> <20020427131213.C1588@eecs.tufts.edu> <3CCB0B56.B910D3E9@prescod.net> <20020427163745.N1588@eecs.tufts.edu> Message-ID: <200204272159.g3RLxpf07396@pcp742651pcs.reston01.va.comcast.net> > Yeah, you're probably right. Hopefully someone knowledgeable on the > subject will enlighten us. I bet you wouldn't be able to detect any performance improvement if the xreadlines() buffer were increased in size. See the comment in the source: /* A larger chunk size doesn't seem to make a difference */ #define CHUNKSIZE 8192 > It does beg the question though: Why isn't > readline mmap'd? Would that be more efficient? (a) File objects are not just used for disk files, but also for sockets (via socket.makefile()), pipes, etc. (b) Mmap APIs vary widely across systems -- we've only got mmap support on Unix and Windows. All this can be addressed by rewriting the I/O system, but that's a huge job that nobody has volunteered to tackle yet. (It's particularly painful because it has to work cross-platform, and beyond stdio there's nothing we can rely on cross-platform.) --Guido van Rossum (home page: http://www.python.org/~guido/) From nhodgson@bigpond.net.au Sat Apr 27 23:09:13 2002 From: nhodgson@bigpond.net.au (Neil Hodgson) Date: Sun, 28 Apr 2002 08:09:13 +1000 Subject: [Python-Dev] _PyString_Resize References: Message-ID: <004d01c1ee38$2a47eaa0$0acc8490@neil> Tim Peters: > The error is that if _PyString_Resize runs out of memory, it automatically > decrefs the input string object s ... Do you have a good way to test out-of-memory handling with either reproducible or random failures? Some of the time I'm conscientious about this and at other times not. Without being able to simulate memory exhaustion the handling code never gets tested and so will be full of bugs. Neil From mgilfix@eecs.tufts.edu Sun Apr 28 00:15:08 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Sat, 27 Apr 2002 19:15:08 -0400 Subject: [Python-Dev] Re: Integrating timeoutsocket.py In-Reply-To: <3CCAF1A7.CBB56B3E@3captus.com>; from bernie@3captus.com on Sat, Apr 27, 2002 at 12:44:55PM -0600 References: <20020419204934.I29377@eecs.tufts.edu> <3CC1323A.7222DB43@3captus.com> <20020425001356.D6826@eecs.tufts.edu> <3CC79357.D6F7EA15@3captus.com> <20020425013009.E6826@eecs.tufts.edu> <3CC7A778.29AFA461@3captus.com> <20020425105117.A19872@eecs.tufts.edu> <3CC902C5.E4BBA660@3captus.com> <20020426184143.K18749@eecs.tufts.edu> <3CCAF1A7.CBB56B3E@3captus.com> Message-ID: <20020427191508.T1588@eecs.tufts.edu> Gonna carbon the dev list to get people's opinions on this one. Something definitely needs to change in the way ServerSocket.py is implemented. Found the cause in ServerSocket.py. You need to add the line: t.setDaemon (1) at line 469 in ServerSocket.py, right after the thread is created. Whether this is an error is debatable though. The problem is that the other threads don't know you're exiting, so they continue running after you exit. Ideally, we should either just set the client threads as daemon threads or we should have some way of signaling them that they must terminate. One solution without touching ServerSocket is to subclass off of ThreadingMixIn and create a new ThreadingTCPServer class. Not too elegant though. The docs for the thread module say: "Threads interact strangely with interrupts: the KeyboardInterrupt exception will be received by an arbitrary thread. (When the signal module is available, interrupts always go to the main thread.)" Kinda unfortunate in this scenario. -- Mike On Sat, Apr 27 @ 12:44, Bernard Yue wrote: > About the test. I am facing this problem with Threading server (using > SocketServer module). > The new socket module somehow change the behavior of the threading > server. The change happens during server exit. > > With the old module, when there is are client(s) connected to the > server, existing the server by KeyboardInterrupt will close the > connection with all the connecting client. Not the case for the new > module. The thread which serving a connected client will stays on until > the client exit. > > I am not exactly sure why it happens. Attached is the script, use > telnet client to do the test and see if you can figure out why. > > > > Bernie > =================================================================== > #!/usr/bin/env python '#!/home/bernie/src/python23/dist/src/python' import time import select import SocketServer class RequestHandler( SocketServer.StreamRequestHandler): def __init__(self, request, client_address, server): SocketServer.StreamRequestHandler.__init__(self, request, client_address, server) def handle( self): """Handle a client""" _remote_id = '%s:%s' % self.request.getpeername() print 'Client %s connected' % _remote_id self.wfile.write( 'Hello %s\n' % _remote_id) while 1: select.select( [self.rfile.fileno()], [], []) try: _in = self.rfile.readline() if not self.process_command( _in, _remote_id): break except IOError: break # connection reset by requester print '%s quited' % _remote_id def process_command( self, command, client='anonymous'): _command = command.strip() if _command == 'exit' or not command: return 0 if _command: print 'Request from %s: [%s]' % (client, _command) self.wfile.write( 'Command [%s] received!\n' % _command) self.wfile.flush() return 1 class MyTCPServer( SocketServer.TCPServer): def __init__( self, server_address, RequestHandlerClass, timeout=None): self.__timeout = timeout SocketServer.TCPServer.__init__( self, server_address, RequestHandlerClass) ''' # set timeout if self.__timeout: try: print 'Set timeout to %s second(s)' % self.socket.gettimeout() print 'Current time is %s' % time.asctime() self.socket.settimeout( self.__timeout) except AttributeError: print 'No timeout with socket' ''' class ThreadingTCPServer( SocketServer.ThreadingMixIn, MyTCPServer): pass def main( argv): if len(argv) == 1: print 'Usage server.py ' return 0 _address = ( argv[1], int(argv[2])) _server = ThreadingTCPServer( _address, RequestHandler, 3) try: _server.serve_forever() except KeyboardInterrupt: _server.server_close() print "Server Terminated!" if __name__ == "__main__": import sys main( sys.argv) -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From martin@v.loewis.de Sun Apr 28 00:19:52 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 28 Apr 2002 01:19:52 +0200 Subject: [Python-Dev] CHUNKSIZE In-Reply-To: <20020427131213.C1588@eecs.tufts.edu> References: <3CCADAC9.CE349EEC@prescod.net> <20020427131213.C1588@eecs.tufts.edu> Message-ID: Michael Gilfix writes: > This comment is more of a question.. Does the chunksize not have > to do with how the filesystem stores blocks? Yes, but probably in non-obvious ways. Many file systems attempt to allocate subsequent blocks in a file into subsequent sectors of the disk. As a result, you can get lots of data in a single rotation of the disk. *If* you are going to use all the data, it is better to transfer them all in one chunk - the kernel needs to setup IO only for a single operation, which then completes at the burst speed of the device. If the disk repositions heads between reads, performance goes down. Regards, Martin From martin@v.loewis.de Sun Apr 28 00:28:09 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 28 Apr 2002 01:28:09 +0200 Subject: [Python-Dev] Unicode entities in XML cause problems :-( In-Reply-To: <20020427213057.F30164@noris.de> References: <20020427213057.F30164@noris.de> Message-ID: "Matthias Urlichs" writes: > >>> import xml.dom.minidom as md > >>> d=md.parseString("bߐ")) > >>> d.writexml(sys.stdout) > ... > UnicodeError: ASCII encoding error: ordinal not in range(128) [...] > Thus, my proposal (which I'm going to implement since I need it...) is to > write such a codec. For simplicity, I propose to accept ü and € > and friends, but to emit them as Ӓ (or whatever). The proper fix, IMO, is to have writexml accept an encoding argument, and, by default, write the output as UTF-8. Then there is no need for character or entity references. In any case, emitting ü and € in XML is wrong: you cannot use them unless your document type provides them - you should not assume that all XML files use the HTML DTD. > After this codec is written, all occurrences of string.replace('&','&') > (and vice versa) within the standard library can be replaced with the > appropriate encode/decode methods. Please see http://python.org/sf/432401. Walter is working on such a codec. Regards, Martin From tim.one@comcast.net Sun Apr 28 02:42:59 2002 From: tim.one@comcast.net (Tim Peters) Date: Sat, 27 Apr 2002 21:42:59 -0400 Subject: [Python-Dev] _PyString_Resize In-Reply-To: <004d01c1ee38$2a47eaa0$0acc8490@neil> Message-ID: [Neil Hodgson] > Do you have a good way to test out-of-memory handling with either > reproducible or random failures? Not really. If I have time, when I fix something like that I'll get into a debugger and force a NULL result from a memory allocator, to exercise the new code. That doesn't prevent bit-rot later, though. > Some of the time I'm conscientious about this and at other times not. > Without being able to simulate memory exhaustion the handling code > never gets tested and so will be full of bugs. Sure. Luckily, they're on paths that should never get executed anyway . Cute: for the heck of it, I just tried adding if ((serialno & 0x3ff) == 0) return NULL; near the start of pymalloc's debug-mode malloc and realloc routines. That instantly caused a segfault, pointing at: PyObject * _PyObject_GC_New(PyTypeObject *tp) { PyObject *op = _PyObject_GC_Malloc(_PyObject_SIZE(tp)); return PyObject_INIT(op, tp); } The problem is that PyObject_INIT *assumes* its first argument is non-NULL, and code that can't guarantee that shouldn't even be using it. So I'll fix that, but I've got no way to add a test to ensure that someone cleverer than I doesn't break it again. From tim.one@comcast.net Sun Apr 28 04:41:27 2002 From: tim.one@comcast.net (Tim Peters) Date: Sat, 27 Apr 2002 23:41:27 -0400 Subject: [Python-Dev] Use for enumerate() In-Reply-To: <20020427102300.GG22555@devel.trillke> Message-ID: [holger krekel] > ... > But anyway, isn't reading and counting still a faster technique than > using readlines() esp. for larger files? How should I know ? readlines() *is* reading and counting under the covers, but at C speed. So the question is whether and when you can program correct chunking at Python speed that's faster than the time readlines() takes to build line objects into a list. Try it and time it. From tim.one@comcast.net Sun Apr 28 05:07:10 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 28 Apr 2002 00:07:10 -0400 Subject: [Python-Dev] Fun with numbers In-Reply-To: <200204260102.g3Q12r927686@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Barry] > >>> 23000 .__class__ = bool > Debug memory block at address p=0x814f544: > 485823496 bytes originally requested > the 4 pad bytes at p-4 are not all FORBIDDENBYTE (0xfb): > at p-4: 0x7a *** OUCH > at p-3: 0x61 *** OUCH > at p-2: 0xc8 *** OUCH > at p-1: 0x3c *** OUCH > the 4 pad bytes at tail=0x250a094c are Segmentation fault Darn. Of the 3 memory corruption problems the debug pymalloc has caught so far, 2 were detected as *under*writes (scribbling before the requested space, as opposed to overwrites scribbling after it). More, they're terrible underwrites, so bad that they corrupt not only the 4 pad bytes before the start, but also the 4 "number of bytes originally requested" bytes before those. That leads to an insane "bytes originally requested" message, and also to a segfault as the checking routine adds that field to the start address and tries to look at the trailing pad bytes. All of this surprises me. I *expected* most corruption to be via overwrites, and I asked everyone before to limit their corruption to no more than 4 bytes . From smurf@noris.de Sun Apr 28 05:16:10 2002 From: smurf@noris.de (Matthias Urlichs) Date: Sun, 28 Apr 2002 06:16:10 +0200 Subject: [Python-Dev] Unicode entities in XML cause problems :-( In-Reply-To: ; from martin@v.loewis.de on Sun, Apr 28, 2002 at 01:28:09AM +0200 References: <20020427213057.F30164@noris.de> Message-ID: <20020428061610.G30164@noris.de> Hi, Martin v. Loewis: > The proper fix, IMO, is to have writexml accept an encoding argument, > and, by default, write the output as UTF-8. Then there is no need for > character or entity references. > The encoding should probably default to the one from the document header (UTF-8 if that isn't given). > In any case, emitting ü and € in XML is wrong: you cannot > use them unless your document type provides them - you should not > assume that all XML files use the HTML DTD. > Good point. On the other hand, I didn't plan to do that anyway. ;-) (Are Ӓ and friends OK with any DTD?) > Please see http://python.org/sf/432401. Walter is working on such a > codec. > Thank you. For XML escaping, the approach suggested by this patch would be to use xmlcharrefreplace() (see the test script) as the error handler. But that doesn't help with &<>". Personally, I rather dislike having to do a separate replace() for these. One approach would be to use character maps which have strategic holes where & < > and possibly " live..? -- Matthias Urlichs | noris network AG | http://smurf.noris.de/ From tim.one@comcast.net Sun Apr 28 05:36:52 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 28 Apr 2002 00:36:52 -0400 Subject: [Python-Dev] Use for enumerate() In-Reply-To: <000b01c1ee0f$e6f94320$1eea7ad1@othello> Message-ID: >>>>> Challenge 3: do it faster and with less code. >>> it should work even if the file is too large >>> to fit in memory (as long as each individual line fits in memory). [Raymond Hettinger] > def getline(filename, lineno): > if lineno < 1: > return '' > f = open(filename) > g = f.readline > return (reduce(lambda x,y: g(), range(lineno), ''), f.close())[0] This is shorter than Guido's, but you don't imagine it's faster, right? Calling a Python-level function for each line read is more expensive than a few measly trips around the eval loop for each line read, even if the latter has to build a two-tuple each time. Guido's comes in at about 4.8 seconds on my test, and this version at about 5.5. I believe my first stab is still the quickest correct version we've seen, at about 2.9. From andymac@bullseye.apana.org.au Sun Apr 28 03:07:53 2002 From: andymac@bullseye.apana.org.au (Andrew MacIntyre) Date: Sun, 28 Apr 2002 13:07:53 +1100 (edt) Subject: [Python-Dev] timeoutsocket patch In-Reply-To: <20020426184623.M18749@eecs.tufts.edu> Message-ID: On Fri, 26 Apr 2002, Michael Gilfix wrote: > Oh yeah, and could a few brave souls volunteer to do some > testing on MS and MAC? The patch is complete at this point > but needs some testin'. > > On Fri, Apr 26 @ 18:05, Michael Gilfix wrote: > > I'm working with Bernard Yue to implement a timeout socket > > patch. Our question: is it fair to assume that any OS that > > supports sockets also supports select? Or do have to have > > added checks? A pointer to the patch please. I'd like to see how the OS/2 EMX port fares... -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au | Snail: PO Box 370 andymac@pcug.org.au | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From bernie@3captus.com Sun Apr 28 06:31:06 2002 From: bernie@3captus.com (Bernard Yue) Date: Sat, 27 Apr 2002 23:31:06 -0600 Subject: [Python-Dev] timeoutsocket patch References: Message-ID: <3CCB891A.4E60C493@3captus.com> Andrew MacIntyre wrote: > > On Fri, 26 Apr 2002, Michael Gilfix wrote: > > > Oh yeah, and could a few brave souls volunteer to do some > > testing on MS and MAC? The patch is complete at this point > > but needs some testin'. > > > > On Fri, Apr 26 @ 18:05, Michael Gilfix wrote: > > > I'm working with Bernard Yue to implement a timeout socket > > > patch. Our question: is it fair to assume that any OS that > > > supports sockets also supports select? Or do have to have > > > added checks? > > A pointer to the patch please. I'd like to see how the OS/2 EMX port > fares... > You can get the source from http://www.3captus.com/Downloads. BTW, does OS/2 support select() on sockets? Bernie > -- > Andrew I MacIntyre "These thoughts are mine alone..." > E-mail: andymac@bullseye.apana.org.au | Snail: PO Box 370 > andymac@pcug.org.au | Belconnen ACT 2616 > Web: http://www.andymac.org/ | Australia > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev -- There are three schools of magic. One: State a tautology, then ring the changes on its corollaries; that's philosophy. Two: Record many facts. Try to find a pattern. Then make a wrong guess at the next fact; that's science. Three: Be aware that you live in a malevolent Universe controlled by Murphy's Law, sometimes offset by Brewster's Factor; that's engineering. -- Robert A. Heinlein From bernie@3captus.com Sun Apr 28 06:56:25 2002 From: bernie@3captus.com (Bernard Yue) Date: Sat, 27 Apr 2002 23:56:25 -0600 Subject: [Python-Dev] Re: Integrating timeoutsocket.py References: <20020419204934.I29377@eecs.tufts.edu> <3CC1323A.7222DB43@3captus.com> <20020425001356.D6826@eecs.tufts.edu> <3CC79357.D6F7EA15@3captus.com> <20020425013009.E6826@eecs.tufts.edu> <3CC7A778.29AFA461@3captus.com> <20020425105117.A19872@eecs.tufts.edu> <3CC902C5.E4BBA660@3captus.com> <20020426184143.K18749@eecs.tufts.edu> <3CCAF1A7.CBB56B3E@3captus.com> <20020427191508.T1588@eecs.tufts.edu> Message-ID: <3CCB8F09.9FB9F8E3@3captus.com> Michael Gilfix wrote: > > Gonna carbon the dev list to get people's opinions on this > one. Something definitely needs to change in the way ServerSocket.py > is implemented. > > Found the cause in ServerSocket.py. You need to add the line: > > t.setDaemon (1) > This looks like a simple and decent proposal. When you do a keyboard interrupt over the server you are intended to interrupt the main thread (in this situation under Linux the interrupt always sends to the main thread). I think you should just summit a patch. At worst it will just be rejected. Under python 2.1, the keyboard interrupt will trigger an exception 'error' in one of the thread and the whole program exit (so serve the propose of interrupt). Behavior actually changed to current status since python 2.2. Is the change by design or by accident? > at line 469 in ServerSocket.py, right after the thread is > created. Whether this is an error is debatable though. The problem > is that the other threads don't know you're exiting, so they > continue running after you exit. Ideally, we should either just set > the client threads as daemon threads or we should have some way of > signaling them that they must terminate. One solution without touching > ServerSocket is to subclass off of ThreadingMixIn and create a new > ThreadingTCPServer class. Not too elegant though. > > The docs for the thread module say: > > "Threads interact strangely with interrupts: the KeyboardInterrupt > exception will be received by an arbitrary thread. (When the signal > module is available, interrupts always go to the main thread.)" > > Kinda unfortunate in this scenario. > > -- Mike > > On Sat, Apr 27 @ 12:44, Bernard Yue wrote: > > About the test. I am facing this problem with Threading server (using > > SocketServer module). > > The new socket module somehow change the behavior of the threading > > server. The change happens during server exit. > > > > With the old module, when there is are client(s) connected to the > > server, existing the server by KeyboardInterrupt will close the > > connection with all the connecting client. Not the case for the new > > module. The thread which serving a connected client will stays on until > > the client exit. > > > > I am not exactly sure why it happens. Attached is the script, use > > telnet client to do the test and see if you can figure out why. > > > > > > > > Bernie > > =================================================================== > > > #!/usr/bin/env python > '#!/home/bernie/src/python23/dist/src/python' > > import time > import select > import SocketServer > > class RequestHandler( SocketServer.StreamRequestHandler): > def __init__(self, request, client_address, server): > SocketServer.StreamRequestHandler.__init__(self, > request, client_address, server) > > def handle( self): > """Handle a client""" > _remote_id = '%s:%s' % self.request.getpeername() > print 'Client %s connected' % _remote_id > self.wfile.write( 'Hello %s\n' % _remote_id) > > while 1: > select.select( [self.rfile.fileno()], [], []) > try: > _in = self.rfile.readline() > if not self.process_command( _in, _remote_id): > break > except IOError: > break # connection reset by requester > > print '%s quited' % _remote_id > > def process_command( self, command, client='anonymous'): > _command = command.strip() > if _command == 'exit' or not command: > return 0 > > if _command: > print 'Request from %s: [%s]' % (client, _command) > self.wfile.write( 'Command [%s] received!\n' % _command) > self.wfile.flush() > > return 1 > > class MyTCPServer( SocketServer.TCPServer): > def __init__( self, server_address, RequestHandlerClass, timeout=None): > self.__timeout = timeout > SocketServer.TCPServer.__init__( self, server_address, > RequestHandlerClass) > > ''' > # set timeout > if self.__timeout: > try: > print 'Set timeout to %s second(s)' % self.socket.gettimeout() > print 'Current time is %s' % time.asctime() > self.socket.settimeout( self.__timeout) > except AttributeError: > print 'No timeout with socket' > ''' > > class ThreadingTCPServer( SocketServer.ThreadingMixIn, MyTCPServer): pass > > def main( argv): > if len(argv) == 1: > print 'Usage server.py ' > return 0 > > _address = ( argv[1], int(argv[2])) > _server = ThreadingTCPServer( _address, RequestHandler, 3) > > try: > _server.serve_forever() > except KeyboardInterrupt: > _server.server_close() > print "Server Terminated!" > > > if __name__ == "__main__": > import sys > main( sys.argv) > > -- > Michael Gilfix > mgilfix@eecs.tufts.edu > > For my gpg public key: > http://www.eecs.tufts.edu/~mgilfix/contact.html > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev -- There are three schools of magic. One: State a tautology, then ring the changes on its corollaries; that's philosophy. Two: Record many facts. Try to find a pattern. Then make a wrong guess at the next fact; that's science. Three: Be aware that you live in a malevolent Universe controlled by Murphy's Law, sometimes offset by Brewster's Factor; that's engineering. -- Robert A. Heinlein From martin@v.loewis.de Sun Apr 28 10:00:53 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 28 Apr 2002 11:00:53 +0200 Subject: [Python-Dev] Re: Integrating timeoutsocket.py In-Reply-To: <20020427191508.T1588@eecs.tufts.edu> References: <20020419204934.I29377@eecs.tufts.edu> <3CC1323A.7222DB43@3captus.com> <20020425001356.D6826@eecs.tufts.edu> <3CC79357.D6F7EA15@3captus.com> <20020425013009.E6826@eecs.tufts.edu> <3CC7A778.29AFA461@3captus.com> <20020425105117.A19872@eecs.tufts.edu> <3CC902C5.E4BBA660@3captus.com> <20020426184143.K18749@eecs.tufts.edu> <3CCAF1A7.CBB56B3E@3captus.com> <20020427191508.T1588@eecs.tufts.edu> Message-ID: Michael Gilfix writes: > Gonna carbon the dev list to get people's opinions on this > one. Something definitely needs to change in the way ServerSocket.py > is implemented. > > Found the cause in ServerSocket.py. You need to add the line: > > t.setDaemon (1) > > at line 469 in ServerSocket.py, right after the thread is > created. Whether this is an error is debatable though. Since this is a semantics change, you need to make this operation optional - either by passing parameters, or by checking state of some object. If you think the setDaemon call should be the default, you can then deprecate the case where the application does not explicitly specify the daemon behaviour (with a deprecation warning (*)), but continue to operate in this case as today - in a future version of Python (say, 2.5), the default can then be changed. Want to work on a patch? Regards, Martin (*) Triggering the deprecation warning just often enough is tricky. In this case, I would trigger it once per implementation class. From ikjos@email.uophx.edu Sun Apr 28 10:23:39 2002 From: ikjos@email.uophx.edu (Ian Kjos) Date: Sun, 28 Apr 2002 04:23:39 -0500 Subject: [Python-Dev] Smart resource destruction (was Re: Use for enumerate) References: <20020427160005.11973.89266.Mailman@mail.python.org><001001c1ee14$0bb2ac20$0dce4542@austin.rr.com> <15562.62131.337533.344976@anthem.wooz.org> Message-ID: <001101c1ee96$6283e280$0dce4542@austin.rr.com> Look, if we're so careful about thread-naive code not crashing the interpretter (*) or circular references leaking memory (**), then I don't see why naive code should be allowed to leak file descriptors, database handles, query buffers, locks, or other similar things which: 1. Are implemented as and provided by C code. 2. Do not contain references to other python objects. (***) 3. Refer to computational resources outside the program's memory. It's not about providing one-shot methods. Clearly better is a function that does everything in one go. What I'm trying to communicate is that knowledge of what (should) happen when a file's notional refcount hits zero should be embedded in the file object, not each bit of code lucky enough to be the last place said file object is used. The same goes for other purely external resources. Basically, we should put things away when we are done with them. The more the language supports the programmer in doing that, the more productive I think she'll be. I also realize that the real resource is the os-level file descriptor, not the language-level file object. (That's more an implementation thing, though.) Now the first character who wants to subclass file() will surely make this scheme all fall apart. I can't think of why to do such a thing, but that doesn't mean some genius (ahem) out there won't. Clearly there are issues. Nobody wants a solution tomorrow. Sometime before Python 3.0 might be nice, but is totally not necessary either. * It doesn't appear possible to write non-thread-naive code in python, or at least not without changing the whole model of computation, which is a black hole that I don't want to think about. That's just as well, though. ** As demonstrated by the loop-savvy garbage collection upgrade, for example. *** .__class__ et. al. notwithstanding. The point is to avoid, for example, circular linked lists. ----- Original Message ----- From: "Barry A. Warsaw" > Here's a 5 minute toy example... > > -Barry > > [big code block....] From martin@v.loewis.de Sun Apr 28 10:16:44 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 28 Apr 2002 11:16:44 +0200 Subject: [Python-Dev] Unicode entities in XML cause problems :-( In-Reply-To: <20020428061610.G30164@noris.de> References: <20020427213057.F30164@noris.de> <20020428061610.G30164@noris.de> Message-ID: "Matthias Urlichs" writes: > > The proper fix, IMO, is to have writexml accept an encoding argument, > > and, by default, write the output as UTF-8. Then there is no need for > > character or entity references. > > > The encoding should probably default to the one from the document header > (UTF-8 if that isn't given). In .toxml, we are going to *create* a document header. We can put anything into there that we want. If you think that the encoding should be the one that the "original" document had - that cannot work. First, the parser does not provide that information, and the DOM does not preserve it. Furthermore, there doesn't even *have* to be an original document - the DOM tree could have been created from scratch. > For XML escaping, the approach suggested by this patch would be to use > xmlcharrefreplace() (see the test script) as the error handler. > But that doesn't help with &<>". Personally, I rather dislike having to do > a separate replace() for these. > > One approach would be to use character maps which have strategic holes > where & < > and possibly " live..? Depends on your output encoding. If you want to use us-ascii as an output encoding, then it would be easy to create a character map codec that has holes for these characters. If the user wants to specify the output encoding, this may be more difficult, since the codec for the output encoding may not be based on character maps. Since this is application that the SF patch has in mind, I doubt you can avoid the replace calls. Regards, Martin From skip@mojam.com Sun Apr 28 13:00:17 2002 From: skip@mojam.com (Skip Montanaro) Date: Sun, 28 Apr 2002 07:00:17 -0500 Subject: [Python-Dev] Weekly Python Bug/Patch Summary Message-ID: <200204281200.g3SC0Hk20158@12-248-41-177.client.attbi.com> Bug/Patch Summary ----------------- 235 open / 2455 total bugs (-2) 99 open / 1449 total patches (+2) New Bugs -------- base64 newlines - documentation (again) (2002-04-22) http://python.org/sf/547037 urllib/httplib vs corrupted tcp/ip stack (2002-04-22) http://python.org/sf/547093 rlcompleter does not expand on [ ] (2002-04-22) http://python.org/sf/547176 Warn for __coerce__ in new-style classes (2002-04-22) http://python.org/sf/547211 cStringIO mangles Unicode (2002-04-23) http://python.org/sf/547537 Build fails in _curses module (2002-04-24) http://python.org/sf/548109 urlparse doesn't handle host?bla (2002-04-24) http://python.org/sf/548176 Fix the METH_CLASS implementation (2002-04-25) http://python.org/sf/548651 os.popen w/o using the shell (2002-04-25) http://python.org/sf/548661 Missing or wrong index entries (2002-04-25) http://python.org/sf/548693 pydoc doesn't show C types (2002-04-25) http://python.org/sf/548845 Lists do have __getitem__ method (2002-04-25) http://python.org/sf/548874 cgitb variable dumps a little flaky (2002-04-26) http://python.org/sf/549038 test_signal hangs -- signal broken on OpenBSD? (2002-04-26) http://python.org/sf/549081 urllib2 POSTs on redirect (2002-04-26) http://python.org/sf/549151 lib-dynload/*.so permissions wrong (2002-04-26) http://python.org/sf/549338 xml.dom.minidom doesn't pass CDATA (2002-04-28) http://python.org/sf/549725 Unicode encoders appears to leak references (2002-04-28) http://python.org/sf/549731 New Patches ----------- Distutils & non-installed Python (2002-04-23) http://python.org/sf/547734 Adds cookie support to urllib2.py (2002-04-24) http://python.org/sf/548197 Floating point issues in body of text (2002-04-26) http://python.org/sf/548943 ConfigParser: optional section header (2002-04-26) http://python.org/sf/549037 RFC 2231 support for email package (2002-04-26) http://python.org/sf/549133 warn on assignment to None, True, False (2002-04-26) http://python.org/sf/549213 iterzip() implementation (2002-04-27) http://python.org/sf/549662 Closed Bugs ----------- cStringIO rejects Unicode strings (2000-10-08) http://python.org/sf/216388 Emacs Python mode bugs (2001-02-14) http://python.org/sf/232398 python-mode indentation bug (2001-08-16) http://python.org/sf/451841 Emacs: python-mode.el (2001-10-22) http://python.org/sf/473623 urllib problems (2002-01-30) http://python.org/sf/511073 bsddb: enable dbopen (file==NULL) (2002-02-07) http://python.org/sf/514433 test_email fails on Mac in summer (2002-04-07) http://python.org/sf/540720 Memory leak with stackframes + inspect (2002-04-12) http://python.org/sf/543148 gcc warning in Modules/readline.c (2002-04-15) http://python.org/sf/544265 Internet Config -666 error (2002-04-17) http://python.org/sf/545059 walk() walks aliases (2002-04-17) http://python.org/sf/545388 py-execute-string doesn't work (2002-04-17) http://python.org/sf/545436 cannot save buffer (2002-04-18) http://python.org/sf/545655 IDLE calltips cause application error (2002-04-19) http://python.org/sf/546078 readlines() bombs - last line incomplete (2002-04-19) http://python.org/sf/546117 load_false/true() not used (2002-04-19) http://python.org/sf/546156 Closed Patches -------------- Emacs auto-detect J/Python mode (2002-01-29) http://python.org/sf/510288 {a,b} in fnmatch.translate (2002-04-17) http://python.org/sf/545150 interactive help in python-mode (2002-04-17) http://python.org/sf/545439 make test_linuxaudiodev ignore EBUSY (2002-04-17) http://python.org/sf/545486 patch for 514433 bsddb.dbopen (NULL) (2002-04-18) http://python.org/sf/545523 From gustav@morpheus.demon.co.uk Sun Apr 28 13:40:20 2002 From: gustav@morpheus.demon.co.uk (Paul Moore) Date: Sun, 28 Apr 2002 13:40:20 +0100 Subject: [Python-Dev] Built in objects supporting slices Message-ID: <3uqncusdqo1gs2tvvs7v878ki8ftsqbmt9@4ax.com> At the moment, built in objects such as lists don't support the slice notaton l[a:b:c]. Would there be support for including this in Python 2.3? If so, what would be the necessary procedure? Would this require a PEP? Alex Martelli has offered to do the code, if someone will champion the work. I'm happy to do this, but I don't have commit privileges, so it would require someone else to actually commit any changes. I presume that the correct approach would be to submit the changes as a patch to SourceForge? Paul Moore From guido@python.org Sun Apr 28 14:09:30 2002 From: guido@python.org (Guido van Rossum) Date: Sun, 28 Apr 2002 09:09:30 -0400 Subject: [Python-Dev] Re: Integrating timeoutsocket.py In-Reply-To: Your message of "28 Apr 2002 11:00:53 +0200." References: <20020419204934.I29377@eecs.tufts.edu> <3CC1323A.7222DB43@3captus.com> <20020425001356.D6826@eecs.tufts.edu> <3CC79357.D6F7EA15@3captus.com> <20020425013009.E6826@eecs.tufts.edu> <3CC7A778.29AFA461@3captus.com> <20020425105117.A19872@eecs.tufts.edu> <3CC902C5.E4BBA660@3captus.com> <20020426184143.K18749@eecs.tufts.edu> <3CCAF1A7.CBB56B3E@3captus.com> <20020427191508.T1588@eecs.tufts.edu> Message-ID: <200204281309.g3SD9Uh16502@pcp742651pcs.reston01.va.comcast.net> > > Found the cause in ServerSocket.py. You need to add the line: > > > > t.setDaemon (1) > > > > at line 469 in ServerSocket.py, right after the thread is > > created. Whether this is an error is debatable though. > > Since this is a semantics change, you need to make this operation > optional - either by passing parameters, or by checking state of some > object. One option would be to insert a call to a hook method at that point. Define the hook to do nothing in the base class. Theyn your own class can override the hook to call t.setDaemon(1). The hook should probably be called with the thread object (t) as its argument. > If you think the setDaemon call should be the default, you can then > deprecate the case where the application does not explicitly specify > the daemon behaviour (with a deprecation warning (*)), but continue to > operate in this case as today - in a future version of Python (say, > 2.5), the default can then be changed. > > Want to work on a patch? > > Regards, > Martin > > (*) Triggering the deprecation warning just often enough is tricky. In > this case, I would trigger it once per implementation class. The default hook could include a call to warnings.warn(). This would only trigger once per program, but I bet that's a good approximation of once per implementation class. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Sun Apr 28 14:14:53 2002 From: guido@python.org (Guido van Rossum) Date: Sun, 28 Apr 2002 09:14:53 -0400 Subject: [Python-Dev] Smart resource destruction (was Re: Use for enumerate) In-Reply-To: Your message of "Sun, 28 Apr 2002 04:23:39 CDT." <001101c1ee96$6283e280$0dce4542@austin.rr.com> References: <20020427160005.11973.89266.Mailman@mail.python.org><001001c1ee14$0bb2ac20$0dce4542@austin.rr.com> <15562.62131.337533.344976@anthem.wooz.org> <001101c1ee96$6283e280$0dce4542@austin.rr.com> Message-ID: <200204281314.g3SDErI16767@pcp742651pcs.reston01.va.comcast.net> > Look, if we're so careful about thread-naive code not crashing the > interpretter (*) or circular references leaking memory (**), then I don't > see why naive code should be allowed to leak file descriptors, database > handles, query buffers, locks, or other similar things which: Ian, why the speech? If you leave the f.close() call out, it will happen automatically. In CPython, it happens automatically when the refcount goes to 0, which (in this example) is when the function returns. In Jython, however, it would be when the garbage collector feels like it, which may be too late in programs that open lots of files -- and that's why we recommend as a general measure to explicitly close files. > *** .__class__ et. al. notwithstanding. The point is to avoid, for example, > circular linked lists. We're way ahead of you here, too. Since Python 2.0, there's GC (in addition to the refcounting) that clears up cycles. :) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Sun Apr 28 14:20:29 2002 From: guido@python.org (Guido van Rossum) Date: Sun, 28 Apr 2002 09:20:29 -0400 Subject: [Python-Dev] Built in objects supporting slices In-Reply-To: Your message of "Sun, 28 Apr 2002 13:40:20 BST." <3uqncusdqo1gs2tvvs7v878ki8ftsqbmt9@4ax.com> References: <3uqncusdqo1gs2tvvs7v878ki8ftsqbmt9@4ax.com> Message-ID: <200204281320.g3SDKTr16816@pcp742651pcs.reston01.va.comcast.net> > At the moment, built in objects such as lists don't support the slice > notaton l[a:b:c]. Would there be support for including this in Python > 2.3? If so, what would be the necessary procedure? Would this require a > PEP? > > Alex Martelli has offered to do the code, if someone will champion the > work. I'm happy to do this, but I don't have commit privileges, so it > would require someone else to actually commit any changes. I presume > that the correct approach would be to submit the changes as a patch to > SourceForge? Yes. No PEP is needed, but I'm -0 on the idea -- I don't see what use there is for this besides theoretical neatness; I'm sure it complicates a lot of code. IOW What's the use case? --Guido van Rossum (home page: http://www.python.org/~guido/) From python@rcn.com Sun Apr 28 15:23:22 2002 From: python@rcn.com (Raymond Hettinger) Date: Sun, 28 Apr 2002 10:23:22 -0400 Subject: [Python-Dev] iterzip() Message-ID: <000901c1eec0$410a2540$7cec7ad1@othello> What do you guys think about the merits of an iterzip() function that behaves like zip() but returns an iterator instead of a list? Raymond Hettinger P.S. There is a patch for experimentation at http://www.python.org/sf/549662 From mgilfix@eecs.tufts.edu Sun Apr 28 16:15:13 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Sun, 28 Apr 2002 11:15:13 -0400 Subject: [Python-Dev] Re: Integrating timeoutsocket.py In-Reply-To: <200204281309.g3SD9Uh16502@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Sun, Apr 28, 2002 at 09:09:30AM -0400 References: <3CC79357.D6F7EA15@3captus.com> <20020425013009.E6826@eecs.tufts.edu> <3CC7A778.29AFA461@3captus.com> <20020425105117.A19872@eecs.tufts.edu> <3CC902C5.E4BBA660@3captus.com> <20020426184143.K18749@eecs.tufts.edu> <3CCAF1A7.CBB56B3E@3captus.com> <20020427191508.T1588@eecs.tufts.edu> <200204281309.g3SD9Uh16502@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020428111513.W1588@eecs.tufts.edu> Sure. I'll work on this patch. Should I log the bug into SF tracker as well? I agree that it would be nice to have some sort of overridden method at that point. The decision just has to be whether or not t.setDaemon (1) should be the default. The warning is a good idea. I think the setDeamon default is what previous users expect. -- Mike On Sun, Apr 28 @ 09:09, Guido van Rossum wrote: > > > Found the cause in ServerSocket.py. You need to add the line: > > > > > > t.setDaemon (1) > > > > > > at line 469 in ServerSocket.py, right after the thread is > > > created. Whether this is an error is debatable though. > > > > Since this is a semantics change, you need to make this operation > > optional - either by passing parameters, or by checking state of some > > object. > > One option would be to insert a call to a hook method at that point. > Define the hook to do nothing in the base class. Theyn your own class > can override the hook to call t.setDaemon(1). The hook should > probably be called with the thread object (t) as its argument. > > > If you think the setDaemon call should be the default, you can then > > deprecate the case where the application does not explicitly specify > > the daemon behaviour (with a deprecation warning (*)), but continue to > > operate in this case as today - in a future version of Python (say, > > 2.5), the default can then be changed. > > > > Want to work on a patch? > > > > Regards, > > Martin > > > > (*) Triggering the deprecation warning just often enough is tricky. In > > this case, I would trigger it once per implementation class. > > The default hook could include a call to warnings.warn(). This would > only trigger once per program, but I bet that's a good approximation > of once per implementation class. :-) > > --Guido van Rossum (home page: http://www.python.org/~guido/) `-> (guido) -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From guido@python.org Sun Apr 28 16:38:52 2002 From: guido@python.org (Guido van Rossum) Date: Sun, 28 Apr 2002 11:38:52 -0400 Subject: [Python-Dev] Re: Integrating timeoutsocket.py In-Reply-To: Your message of "Sun, 28 Apr 2002 11:15:13 EDT." <20020428111513.W1588@eecs.tufts.edu> References: <3CC79357.D6F7EA15@3captus.com> <20020425013009.E6826@eecs.tufts.edu> <3CC7A778.29AFA461@3captus.com> <20020425105117.A19872@eecs.tufts.edu> <3CC902C5.E4BBA660@3captus.com> <20020426184143.K18749@eecs.tufts.edu> <3CCAF1A7.CBB56B3E@3captus.com> <20020427191508.T1588@eecs.tufts.edu> <200204281309.g3SD9Uh16502@pcp742651pcs.reston01.va.comcast.net> <20020428111513.W1588@eecs.tufts.edu> Message-ID: <200204281538.g3SFcqw16897@pcp742651pcs.reston01.va.comcast.net> > Sure. I'll work on this patch. Should I log the bug into SF tracker > as well? I agree that it would be nice to have some sort of overridden > method at that point. The decision just has to be whether or not > t.setDaemon (1) should be the default. The warning is a good idea. I > think the setDeamon default is what previous users expect. I have to admit I'm pretty fuzzy on the details. Running a server interactively and hitting ^C doesn't seem like a very common use case to me. I'm not sure the setDaemon() is right in other situations -- if a server decides to exit gracefully, shouldn't it let a client session run to completion? --Guido van Rossum (home page: http://www.python.org/~guido/) From mgilfix@eecs.tufts.edu Sun Apr 28 16:42:11 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Sun, 28 Apr 2002 11:42:11 -0400 Subject: [Python-Dev] Re: Integrating timeoutsocket.py In-Reply-To: <200204281538.g3SFcqw16897@pcp742651pcs.reston01.va.comcast.net>; from guido@python.org on Sun, Apr 28, 2002 at 11:38:52AM -0400 References: <3CC7A778.29AFA461@3captus.com> <20020425105117.A19872@eecs.tufts.edu> <3CC902C5.E4BBA660@3captus.com> <20020426184143.K18749@eecs.tufts.edu> <3CCAF1A7.CBB56B3E@3captus.com> <20020427191508.T1588@eecs.tufts.edu> <200204281309.g3SD9Uh16502@pcp742651pcs.reston01.va.comcast.net> <20020428111513.W1588@eecs.tufts.edu> <200204281538.g3SFcqw16897@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020428114211.X1588@eecs.tufts.edu> Well, let's take a common restart scenario such as a SIGHUP or something along those lines. The server has to restart and load a new configuration file... Then I guess keeping client connections open is a matter of policy. I'm just worried that somehow, someone structured the server such that the clients rely on the main thread somehow. Hmm. I guess it's easier just to default to it off. Makes more sense to think of enabling a feature when subclassing anyway. On Sun, Apr 28 @ 11:38, Guido van Rossum wrote: > I have to admit I'm pretty fuzzy on the details. Running a server > interactively and hitting ^C doesn't seem like a very common use case > to me. I'm not sure the setDaemon() is right in other situations -- > if a server decides to exit gracefully, shouldn't it let a client > session run to completion? > > --Guido van Rossum (home page: http://www.python.org/~guido/) `-> (guido) -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From aleax@aleax.it Sun Apr 28 17:53:47 2002 From: aleax@aleax.it (Alex Martelli) Date: Sun, 28 Apr 2002 18:53:47 +0200 Subject: [Python-Dev] Built in objects supporting slices In-Reply-To: <200204281320.g3SDKTr16816@pcp742651pcs.reston01.va.comcast.net> References: <3uqncusdqo1gs2tvvs7v878ki8ftsqbmt9@4ax.com> <200204281320.g3SDKTr16816@pcp742651pcs.reston01.va.comcast.net> Message-ID: <02042818534700.24988@arthur> On Sunday 28 April 2002 15:20, Guido van Rossum wrote: > > At the moment, built in objects such as lists don't support the slice > > notaton l[a:b:c]. Would there be support for including this in Python ... > Yes. No PEP is needed, but I'm -0 on the idea -- I don't see what use > there is for this besides theoretical neatness; I'm sure it > complicates a lot of code. > > IOW What's the use case? If I understood the c.l.py discussion correctly, one key issue is with documentation -- currently, in: http://www.python.org/dev/doc/devel/ref/sequence-types.html the "right" way for a contained object to support slicing is said to be through __getitem__ (with a slice argument), while __getslice__ is said to be supported only for backwards compatibility. If you think __getslice__ is best, then that page should be edited accordingly; otherwise, built-in sequences should probably respect the same "right way" as others. The only use-case I would probably use substantially is a step of -1 for reverse looping. I do see in my existing code some number of: "for i in range(len(L)-1, 0, -1):" followed by use of L[i], and the like, that would be "smoother" as "for item in L[::-1]:". Not a huge win, for sure, but not terrible, it would seem to me. I see no such immediate application for other steps, although Python's standard libraries do have some cases with a step of 2 that might use a step-2 slice. So, the benefits are admittedly modest, but, it seems to me, > 0. Alex From python@rcn.com Sun Apr 28 18:10:09 2002 From: python@rcn.com (Raymond Hettinger) Date: Sun, 28 Apr 2002 13:10:09 -0400 Subject: [Python-Dev] Built in objects supporting slices Message-ID: <001701c1eed7$8deec7a0$2066accf@othello> > From: Paul Moore > > At the moment, built in objects such as lists don't support the slice > notaton l[a:b:c]. Would there be support for including this in Python > 2.3? If so, what would be the necessary procedure? Would this require a > PEP? > > Alex Martelli has offered to do the code, if someone will champion the > work. I'm happy to do this, but I don't have commit privileges, so it > would require someone else to actually commit any changes. I presume > that the correct approach would be to submit the changes as a patch to > SourceForge? I would be happy to help on this one either in the design stage, code review, or writing the unittests and docs. Maybe we should also address the str/getitem/slice bug www.python.org/sf/473985 and kill two birds with one stone (or tie two swallows to one coconut or some such). Raymond Hettinger From aleax@aleax.it Sun Apr 28 20:47:00 2002 From: aleax@aleax.it (Alex Martelli) Date: Sun, 28 Apr 2002 21:47:00 +0200 Subject: [Python-Dev] Built in objects supporting slices In-Reply-To: <001701c1eed7$8deec7a0$2066accf@othello> References: <001701c1eed7$8deec7a0$2066accf@othello> Message-ID: <02042821470001.25256@arthur> On Sunday 28 April 2002 19:10, Raymond Hettinger wrote: ... > Maybe we should also address the str/getitem/slice bug > www.python.org/sf/473985 and kill two birds with one stone (or tie two Yes, GvR's comments there do seem to address most of the work needed to address this proposal, excepting only the details of handling slices with step != 1 -- which aren't the hardest part. Alex From Jack.Jansen@oratrix.com Sun Apr 28 23:05:13 2002 From: Jack.Jansen@oratrix.com (Jack Jansen) Date: Mon, 29 Apr 2002 00:05:13 +0200 Subject: [Python-Dev] Unicode In-Reply-To: <200204261626.g3QGQdI03824@pcp742651pcs.reston01.va.comcast.net> Message-ID: <03B68F31-5AF4-11D6-895F-003065517236@oratrix.com> On vrijdag, april 26, 2002, at 06:26 , Guido van Rossum wrote: > No syntactic changes, no. But the way we do things would become > significantly different. And think of binary I/O vs. textual I/O -- > currently, file.read() returns a string. Code dealing with binary > files will look significantly different, and old code won't work. It could be argued that open(..., 'r').read() returns a text string and open(..., 'rb').read() returns a binary blob. If textstrings and blobs become wholly different objects this shouldn't create too many problems [see below], except for code that opens a file in binary mode and (partially) reads the resulting file expecting text. But this code would need revisiting anyway if the normal textstring would become unicode. [here's below] To my surprise I think that having blobs and textstrings be unrelated objects creates less problems than having the one be a subtype of the other. At least, every time I try to do the subtyping in my head I flip back and forth between textstrings-are-a-subtype-of-general-binary-buffers and binary-buffers-are-a-special-case-of-python-strings every couple of seconds. I think having them both be subtypes of a common base type (basestring) might work, but I'm not sure. -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From guido@python.org Sun Apr 28 23:14:25 2002 From: guido@python.org (Guido van Rossum) Date: Sun, 28 Apr 2002 18:14:25 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Your message of "Sun, 28 Apr 2002 10:23:22 EDT." <000901c1eec0$410a2540$7cec7ad1@othello> References: <000901c1eec0$410a2540$7cec7ad1@othello> Message-ID: <200204282214.g3SMEPI17614@pcp742651pcs.reston01.va.comcast.net> > What do you guys think about the merits of an iterzip() function that > behaves like zip() but returns an iterator instead of a list? Hm, I'm not particularly enamored of the idea of adding 'iter' versions of everything under the sun. I wish zip() could've been an interator from the start, but now that it isn't, I don't think it's such a big deal. (An iterator version is easily written as a generator.) In general I'm not keen on increasing the number of builtin functions much. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Sun Apr 28 23:30:22 2002 From: guido@python.org (Guido van Rossum) Date: Sun, 28 Apr 2002 18:30:22 -0400 Subject: [Python-Dev] Built in objects supporting slices In-Reply-To: Your message of "Sun, 28 Apr 2002 18:53:47 +0200." <02042818534700.24988@arthur> References: <3uqncusdqo1gs2tvvs7v878ki8ftsqbmt9@4ax.com> <200204281320.g3SDKTr16816@pcp742651pcs.reston01.va.comcast.net> <02042818534700.24988@arthur> Message-ID: <200204282230.g3SMUM217981@pcp742651pcs.reston01.va.comcast.net> > On Sunday 28 April 2002 15:20, Guido van Rossum wrote: > > > At the moment, built in objects such as lists don't support the slice > > > notaton l[a:b:c]. Would there be support for including this in Python > ... > > Yes. No PEP is needed, but I'm -0 on the idea -- I don't see what use > > there is for this besides theoretical neatness; I'm sure it > > complicates a lot of code. > > > > IOW What's the use case? > > If I understood the c.l.py discussion correctly, one key issue is > with documentation -- currently, in: > > http://www.python.org/dev/doc/devel/ref/sequence-types.html > > the "right" way for a contained object to support slicing is said > to be through __getitem__ (with a slice argument), while > __getslice__ is said to be supported only for backwards > compatibility. If you think __getslice__ is best, then that page > should be edited accordingly; otherwise, built-in sequences > should probably respect the same "right way" as others. > > The only use-case I would probably use substantially is a step > of -1 for reverse looping. I do see in my existing code some number > of: "for i in range(len(L)-1, 0, -1):" followed by use of L[i], and > the like, that would be "smoother" as "for item in L[::-1]:". Not > a huge win, for sure, but not terrible, it would seem to me. I see > no such immediate application for other steps, although Python's > standard libraries do have some cases with a step of 2 that might > use a step-2 slice. > > So, the benefits are admittedly modest, but, it seems to me, > 0. +1 on making the built-in types accept slice objects in their __getitem__, __setitem__, __delitem__ methods. +0 on subsequently getting rid of their __getslice__, __setslice__, __delslice__ methods. PyObject_GetSlice c.s. may have to be adapted to fall back to __getitem__ with a slice object. The slice opcodes may possibly be simplified. Possible issue: I believe getslice and getitem deal differently with negative indices (one passes them on, the other doesn't, or something like that). There is a bit of a rat's nest trying to keep this compatible in all ways. --Guido van Rossum (home page: http://www.python.org/~guido/) From aleax@aleax.it Sun Apr 28 23:29:39 2002 From: aleax@aleax.it (Alex Martelli) Date: Mon, 29 Apr 2002 00:29:39 +0200 Subject: [Python-Dev] iterzip() In-Reply-To: <200204282214.g3SMEPI17614@pcp742651pcs.reston01.va.comcast.net> References: <000901c1eec0$410a2540$7cec7ad1@othello> <200204282214.g3SMEPI17614@pcp742651pcs.reston01.va.comcast.net> Message-ID: <02042900293903.25256@arthur> On Monday 29 April 2002 00:14, Guido van Rossum wrote: ... > such a big deal. (An iterator version is easily written as a > generator.) > > In general I'm not keen on increasing the number of builtin functions Maybe we could have a standard module in the library that supplies such "utility" iterators and generators? That avoids adding builtin functions but still provides beginners a good sample of the things they can use iterators and generators for. (If it was a separate module, it might not be hard for somebody to supply a version compatible with 2.2, easing migration and cross-version use). Alex From barry@zope.com Sun Apr 28 23:41:43 2002 From: barry@zope.com (Barry A. Warsaw) Date: Sun, 28 Apr 2002 18:41:43 -0400 Subject: [Python-Dev] Unicode References: <200204261626.g3QGQdI03824@pcp742651pcs.reston01.va.comcast.net> <03B68F31-5AF4-11D6-895F-003065517236@oratrix.com> Message-ID: <15564.31399.175643.35918@anthem.wooz.org> >>>>> "JJ" == Jack Jansen writes: JJ> [here's below] To my surprise I think that having blobs and JJ> textstrings be unrelated objects creates less problems than JJ> having the one be a subtype of the other. While I think there's a lot of common operations you might do on blobs and textstrings, I don't see much of a need for them to be related via class hierarchy (although we might find it to be expedient for backwards compatibility). -Barry From barry@zope.com Sun Apr 28 23:45:04 2002 From: barry@zope.com (Barry A. Warsaw) Date: Sun, 28 Apr 2002 18:45:04 -0400 Subject: [Python-Dev] iterzip() References: <000901c1eec0$410a2540$7cec7ad1@othello> <200204282214.g3SMEPI17614@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15564.31600.971790.869889@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> Hm, I'm not particularly enamored of the idea of adding GvR> 'iter' versions of everything under the sun. I wish zip() GvR> could've been an interator from the start, but now that it GvR> isn't, I don't think it's such a big deal. (An iterator GvR> version is easily written as a generator.) Of course, zip() wasn't implemented originally as an iterator because it pre-dated them. I agree, it's probably not worth changing it. -Barry From python@rcn.com Mon Apr 29 01:16:30 2002 From: python@rcn.com (Raymond Hettinger) Date: Sun, 28 Apr 2002 20:16:30 -0400 Subject: [Python-Dev] iterzip() References: <000901c1eec0$410a2540$7cec7ad1@othello> <200204282214.g3SMEPI17614@pcp742651pcs.reston01.va.comcast.net> Message-ID: <001801c1ef13$1d97f6c0$85e77ad1@othello> From: "Guido van Rossum" > Hm, I'm not particularly enamored of the idea of adding 'iter' > versions of everything under the sun. I'm already working on a separate module for iterators galore (and will cross-check to Haskell to make sure I didn't miss anything). I posted this one separately because zip() eats memory like crazy and because a Python generator version crawls like a snail. IMHO, This is a better way to loop over multiple sequences and has a chance at becoming the tool of choice. I scanned all of my Python code and found that iterzip() was a better choice in every case except a matrix transpose coded as zip(*mat). > I wish zip() could've been an > interator from the start, but now that it isn't, I don't think it's > such a big deal. (An iterator version is easily written as a > generator.) > > In general I'm not keen on increasing the number of builtin functions > much. Ditto. Any chance of moving functions like map(), reduce(), and filter() to a functional module; pow() and divmod() to the math module; or input() to oblivion? Raymond Hettinger From guido@python.org Mon Apr 29 01:32:53 2002 From: guido@python.org (Guido van Rossum) Date: Sun, 28 Apr 2002 20:32:53 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Your message of "Sun, 28 Apr 2002 20:16:30 EDT." <001801c1ef13$1d97f6c0$85e77ad1@othello> References: <000901c1eec0$410a2540$7cec7ad1@othello> <200204282214.g3SMEPI17614@pcp742651pcs.reston01.va.comcast.net> <001801c1ef13$1d97f6c0$85e77ad1@othello> Message-ID: <200204290032.g3T0WrU18240@pcp742651pcs.reston01.va.comcast.net> > I'm already working on a separate module for iterators galore > (and will cross-check to Haskell to make sure I didn't miss anything). +! > I posted this one separately because zip() eats memory like crazy > and because a Python generator version crawls like a snail. Do you have use cases where the memory use matters? I.e. where it needs more memory than you have RAM? > IMHO, This is a better way to loop over multiple sequences and > has a chance at becoming the tool of choice. I scanned all of my > Python code and found that iterzip() was a better choice in every > case except a matrix transpose coded as zip(*mat). Did you time any of these? > > In general I'm not keen on increasing the number of builtin functions > > much. > > Ditto. Any chance of moving functions like map(), reduce(), and filter() > to a functional module; pow() and divmod() to the math module; or > input() to oblivion? I wish. Since they were there first, it's hard to get rid of them. (If you're truly masochist, write a PEP and post it to c.l.py to find out how hard. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 29 01:38:42 2002 From: guido@python.org (Guido van Rossum) Date: Sun, 28 Apr 2002 20:38:42 -0400 Subject: [Python-Dev] Unicode In-Reply-To: Your message of "Mon, 29 Apr 2002 00:05:13 +0200." <03B68F31-5AF4-11D6-895F-003065517236@oratrix.com> References: <03B68F31-5AF4-11D6-895F-003065517236@oratrix.com> Message-ID: <200204290038.g3T0cgn18255@pcp742651pcs.reston01.va.comcast.net> [Guido] > > No syntactic changes, no. But the way we do things would become > > significantly different. And think of binary I/O vs. textual I/O -- > > currently, file.read() returns a string. Code dealing with binary > > files will look significantly different, and old code won't work. [Jack] > It could be argued that open(..., 'r').read() returns a text > string and open(..., 'rb').read() returns a binary blob. They might even return different kind of objects -- arguably, binary files don't need readline() etc., and text files may not need read(n) (though the arg-less variant is handy). If only I had the time to reinvent the I/O library... > If textstrings and blobs become wholly different objects this > shouldn't create too many problems [see below], except for code > that opens a file in binary mode and (partially) reads the > resulting file expecting text. But this code would need > revisiting anyway if the normal textstring would become unicode. Yeah, that's usually just stubborn Unix users who don't believe in the distinction between binary and text mode. :-) Anyway, the proper way to convert between blobs and textstrings would be encodings. That's how Java does it. > [here's below] To my surprise I think that having blobs and > textstrings be unrelated objects creates less problems than > having the one be a subtype of the other. At least, every time I > try to do the subtyping in my head I flip back and forth between > textstrings-are-a-subtype-of-general-binary-buffers and > binary-buffers-are-a-special-case-of-python-strings every couple > of seconds. I think having them both be subtypes of a common > base type (basestring) might work, but I'm not sure. I think they don't need anything in common (apart their sequence-ness). I think Java's byte[] vs. String distinction is about right. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 29 01:56:33 2002 From: guido@python.org (Guido van Rossum) Date: Sun, 28 Apr 2002 20:56:33 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Your message of "Mon, 29 Apr 2002 00:29:39 +0200." <02042900293903.25256@arthur> References: <000901c1eec0$410a2540$7cec7ad1@othello> <200204282214.g3SMEPI17614@pcp742651pcs.reston01.va.comcast.net> <02042900293903.25256@arthur> Message-ID: <200204290056.g3T0uXL18431@pcp742651pcs.reston01.va.comcast.net> > Maybe we could have a standard module in the library that > supplies such "utility" iterators and generators? That avoids > adding builtin functions but still provides beginners a good > sample of the things they can use iterators and generators for. > (If it was a separate module, it might not be hard for somebody > to supply a version compatible with 2.2, easing migration and > cross-version use). Yup, that's much less of a problem IMO. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Mon Apr 29 01:56:05 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 28 Apr 2002 20:56:05 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <200204290032.g3T0WrU18240@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Raymond Hettinger] > Ditto. Any chance of moving functions like map(), reduce(), > and filter() to a functional module; pow() and divmod() to the math > module; or input() to oblivion? [Guido] > I wish. Since they were there first, it's hard to get rid of them. > (If you're truly masochist, write a PEP and post it to c.l.py to find > out how hard. :-) If you agree to approve the PEP in advance, it's easy: Raymond posts it, ignores all responses except to summarize the arguments against in the PEP, and you approve it from your beach mansion on the Cayman Islands. BTW, note that math.pow already exists, and isn't the same as the builtin pow. You can agree in advance to break that too . From barry@zope.com Mon Apr 29 02:05:42 2002 From: barry@zope.com (Barry A. Warsaw) Date: Sun, 28 Apr 2002 21:05:42 -0400 Subject: [Python-Dev] iterzip() References: <000901c1eec0$410a2540$7cec7ad1@othello> <200204282214.g3SMEPI17614@pcp742651pcs.reston01.va.comcast.net> <001801c1ef13$1d97f6c0$85e77ad1@othello> Message-ID: <15564.40038.610923.30406@anthem.wooz.org> >>>>> "RH" == Raymond Hettinger writes: RH> IMHO, This is a better way to loop over multiple sequences and RH> has a chance at becoming the tool of choice. I scanned all of RH> my Python code and found that iterzip() was a better choice in RH> every case except a matrix transpose coded as zip(*mat). I'd much rather see a patch that just changed zip() to be an iterator instead of adding an iterzip(). I doubt that much in-field code would break because of it (but write the PEP to find out. ;). RH> Ditto. Any chance of moving functions like map(), reduce(), RH> and filter() to a functional module; pow() and divmod() to the RH> math module; or input() to oblivion? >>>>> "GvR" == Guido van Rossum writes: GvR> I wish. Since they were there first, it's hard to get rid of GvR> them. (If you're truly masochist, write a PEP and post it to GvR> c.l.py to find out how hard. :-) The PEP would have to specify a migration plan, i.e. the builtins are identical to the functional module versions, and would a deprecation schedule, etc. -Barry From python@rcn.com Mon Apr 29 02:39:53 2002 From: python@rcn.com (Raymond Hettinger) Date: Sun, 28 Apr 2002 21:39:53 -0400 Subject: [Python-Dev] iterzip() References: <000901c1eec0$410a2540$7cec7ad1@othello><200204282214.g3SMEPI17614@pcp742651pcs.reston01.va.comcast.net><001801c1ef13$1d97f6c0$85e77ad1@othello> <15564.40038.610923.30406@anthem.wooz.org> Message-ID: <000b01c1ef1e$c39ba7a0$47e97ad1@othello> ----- Original Message ----- From: "Barry A. Warsaw" To: "Raymond Hettinger" Cc: "Guido van Rossum" ; > >>>>> "RH" == Raymond Hettinger writes: > > RH> IMHO, This is a better way to loop over multiple sequences and > RH> has a chance at becoming the tool of choice. I scanned all of > RH> my Python code and found that iterzip() was a better choice in > RH> every case except a matrix transpose coded as zip(*mat). > > I'd much rather see a patch that just changed zip() to be an iterator > instead of adding an iterzip(). I doubt that much in-field code would > break because of it (but write the PEP to find out. ;). I started down this road, by checking code in the Vaults of Parnassus on the assumption that zip() is rarely used outside a for-loop. What may be the killer is the examples of zip() in Python books which demonstrate a stand-alone zip() returning a list -- in some ways, textbook examples are just as important as in-field code. I will write-up a PEP to see if the world cares. For all I know, I may be the only who uses zip() throughout my code. Zip is just new enough that it might not be too late to change it to an iterator. > > RH> Ditto. Any chance of moving functions like map(), reduce(), > RH> and filter() to a functional module; pow() and divmod() to the > RH> math module; or input() to oblivion? > > >>>>> "GvR" == Guido van Rossum writes: > > GvR> I wish. Since they were there first, it's hard to get rid of > GvR> them. (If you're truly masochist, write a PEP and post it to > GvR> c.l.py to find out how hard. :-) > > The PEP would have to specify a migration plan, i.e. the builtins are > identical to the functional module versions, and would a deprecation > schedule, etc. Will do. I think the resistance to moving the functionals will be fierce. Divmod probably has very few instances in real code. I think __pow__ would need to be left in (as the ** that calls it), but the function itself may be used only rarely. Does anyone know of an automated what that I can scan a large body of published Python code. I would want some real usage statistics in a PEP but hate pulling modules down one at a time and grepping them. Raymond Hettinger > > -Barry From barry@zope.com Mon Apr 29 02:47:20 2002 From: barry@zope.com (Barry A. Warsaw) Date: Sun, 28 Apr 2002 21:47:20 -0400 Subject: [Python-Dev] iterzip() References: <000901c1eec0$410a2540$7cec7ad1@othello> <200204282214.g3SMEPI17614@pcp742651pcs.reston01.va.comcast.net> <001801c1ef13$1d97f6c0$85e77ad1@othello> <15564.40038.610923.30406@anthem.wooz.org> <000b01c1ef1e$c39ba7a0$47e97ad1@othello> Message-ID: <15564.42536.646377.149285@anthem.wooz.org> >>>>> "RH" == Raymond Hettinger writes: RH> I started down this road, by checking code in the Vaults of RH> Parnassus on the assumption that zip() is rarely used outside RH> a for-loop. What may be the killer is the examples of zip() RH> in Python books which demonstrate a stand-alone zip() RH> returning a list -- in some ways, textbook examples are just RH> as important as in-field code. RH> I will write-up a PEP to see if the world cares. For all I RH> know, I may be the only who uses zip() throughout my code. RH> Zip is just new enough that it might not be too late to change RH> it to an iterator. The question is how the zip() list is used. E.g. if it just ends up being the source of a for-loop sequence, then it makes little difference in practice. RH> Does anyone know of an automated what that I can scan a large RH> body of published Python code. I would want some real usage RH> statistics in a PEP but hate pulling modules down one at a RH> time and grepping them. No, but it would be a really cool tool. And if it's available via cgi or some other commonly available process, Mark Hammond could add it to his neat Mozilla sidebar. Wild thought: write a cron script which does a cvs update every hour, and runs Tools/scripts/eptags.py over the standard library. Then front a simple TAGS file searcher with cgi. Then maybe we can put it up on python.org's cgi-bin directory. -Barry From tim.one@comcast.net Mon Apr 29 02:59:57 2002 From: tim.one@comcast.net (Tim Peters) Date: Sun, 28 Apr 2002 21:59:57 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <000b01c1ef1e$c39ba7a0$47e97ad1@othello> Message-ID: [Raymond Hettinger] > ... > Will do. I think the resistance to moving the functionals will be > fierce. Divmod probably has very few instances in real code. I think > __pow__ would need to be left in (as the ** that calls it), but the > function itself may be used only rarely. I suggest you don't slash your chances of success by expanding the scope beyond what you really want. Python is a pleasant language for integer number-crunching code, and some people (I happen to be one of them) have hundreds of instances of divmod and 3-argument pow() in their code (note that the latter cannot be spelled via **). People have been asking for "something like" enumerate() for 10 years, and that's the right amount of time to test that it's really wanted . Contrarily, people have only been whining about input() for a couple of years (I use it all the time -- and now that you know that I do, I *dare* you to crack my home box as a result ). This thread is the first time I ever saw anyone complain about divmod(). The confusion about builin pow() vs math.pow vs ** is old but infrequent. -1-on-gratuitous-fiddling-ly y'rs - tim From python@rcn.com Mon Apr 29 04:04:04 2002 From: python@rcn.com (Raymond Hettinger) Date: Sun, 28 Apr 2002 23:04:04 -0400 Subject: [Python-Dev] iterzip() References: <000901c1eec0$410a2540$7cec7ad1@othello> <200204282214.g3SMEPI17614@pcp742651pcs.reston01.va.comcast.net> <001801c1ef13$1d97f6c0$85e77ad1@othello> <200204290032.g3T0WrU18240@pcp742651pcs.reston01.va.comcast.net> Message-ID: <009001c1ef2a$86896260$47e97ad1@othello> From: "Guido van Rossum" > > I'm already working on a separate module for iterators galore > > (and will cross-check to Haskell to make sure I didn't miss anything). > > +! > > > I posted this one separately because zip() eats memory like crazy > > and because a Python generator version crawls like a snail. > > Do you have use cases where the memory use matters? I.e. where it > needs more memory than you have RAM? No. Am I'm not proposing a backport to Pippy ;) > > > IMHO, This is a better way to loop over multiple sequences and > > has a chance at becoming the tool of choice. I scanned all of my > > Python code and found that iterzip() was a better choice in every > > case except a matrix transpose coded as zip(*mat). > > Did you time any of these? I just timed it and am shocked. iterzip() has exactly the same code as zip() except for the final append result to list. So, I expected only a microscopic speed-up. I don't see where the huge performance improvement came from. First timing: ---------------------- 19.439999938 zip 1.43000006676 iterzip 30.1000000238 genzip Second timing: ---------------------- 20.0999999046 zip 1.60000002384 iterzip 29.0599999428 genzip Timing code: --------------- # time iterzip from __future__ import generators import time def run(f, iterables): start = time.time() for tup in f(*iterables): pass return time.time() - start def genzip(*iterables): iterables = map(iter, iterables) while 1: yield tuple([i.next() for i in iterables]) n = 1000000 for f in [zip, iterzip, genzip]: print run(f, [xrange(n), xrange(n), xrange(n)]), f.__name__ Raymond Hettinger From andymac@bullseye.apana.org.au Sun Apr 28 22:12:03 2002 From: andymac@bullseye.apana.org.au (Andrew MacIntyre) Date: Mon, 29 Apr 2002 08:12:03 +1100 (edt) Subject: [Python-Dev] timeoutsocket patch In-Reply-To: <3CCB891A.4E60C493@3captus.com> Message-ID: On Sat, 27 Apr 2002, Bernard Yue wrote: > Andrew MacIntyre wrote: > > > > On Fri, 26 Apr 2002, Michael Gilfix wrote: > > > > > Oh yeah, and could a few brave souls volunteer to do some > > > testing on MS and MAC? The patch is complete at this point > > > but needs some testin'. > > > > > > On Fri, Apr 26 @ 18:05, Michael Gilfix wrote: > > > > I'm working with Bernard Yue to implement a timeout socket > > > > patch. Our question: is it fair to assume that any OS that > > > > supports sockets also supports select? Or do have to have > > > > added checks? > > > > A pointer to the patch please. I'd like to see how the OS/2 EMX port > > fares... > > You can get the source from http://www.3captus.com/Downloads. BTW, does > OS/2 support select() on sockets? Thanks. The EMX runtime library supports select() on sockets; I don't believe that native OS/2 has select() at all, but can't confirm. -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au | Snail: PO Box 370 andymac@pcug.org.au | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From mgilfix@eecs.tufts.edu Mon Apr 29 05:34:14 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Mon, 29 Apr 2002 00:34:14 -0400 Subject: [Python-Dev] timeoutsocket patch In-Reply-To: ; from andymac@bullseye.apana.org.au on Mon, Apr 29, 2002 at 08:12:03AM +1100 References: <3CCB891A.4E60C493@3captus.com> Message-ID: <20020429003413.B12835@eecs.tufts.edu> Hmm, how do you poll sockets on OS/2 then? If there is some way, then maybe that hack should go into the select library (the select library treats sockets specially as polleable objects). Perhaps I should rework the patch so that use of select is avoided if either the import fails or the timeout is None. But (and maybe Guido will offer his opinion here), I'd like to hold out to see if it really is the case that some system doesn't support a select equivalent for sockets. -- Mike On Mon, Apr 29 @ 08:12, Andrew MacIntyre wrote: > Thanks. The EMX runtime library supports select() on sockets; I don't > believe that native OS/2 has select() at all, but can't confirm. -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From tim.one@comcast.net Mon Apr 29 06:05:15 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 29 Apr 2002 01:05:15 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <009001c1ef2a$86896260$47e97ad1@othello> Message-ID: [Raymond Hettinger] > ... > I just timed it and am shocked. iterzip() has exactly the same code > as zip() except for the final append result to list. So, I expected only > a microscopic speed-up. I don't see where the huge performance > improvement came from. Well, in the zip case you're allocating 4 million bytes worth of integers, 4 bytes at a time, and don't get to reuse any of the space for the duration. Likewise you're allocating space for a million 3-element tuples, and 4MB worth of contiguous vector 4 bytes at a time. That's all a real drag. Cute: I can make the Python genzip run faster than the builtin zip with "the usual" speed tricks: from time import clock as now def run(f, iterables): start = now() for tup in f(*iterables): pass return now() - start def genzip(*iterables): getters = [iter(i).next for i in iterables] t = tuple while 1: yield t([g() for g in getters]) n = 1000000 for f in [zip, genzip]: print run(f, [xrange(n), xrange(n), xrange(n)]), f.__name__ On my box under current CVS, that yields C:\Code\python\PCbuild>python -O temp.py 12.3587522419 zip 10.4161551484 genzip Better, I can get genzip down to 8.34 seconds by skipping the tuple() call -- in real life, it doesn't really matter whether it returns a tuple or a list. One reason zip is faster for me than for you is that pymalloc is enabled by default in 2.3, and can likely pump out space for a million small tuples significantly faster than your platform malloc can do it. > First timing: > ---------------------- > 19.439999938 zip > 1.43000006676 iterzip > 30.1000000238 genzip If only I had a use for iterzip . From aleax@aleax.it Mon Apr 29 06:59:05 2002 From: aleax@aleax.it (Alex Martelli) Date: Mon, 29 Apr 2002 07:59:05 +0200 Subject: [Python-Dev] iterzip 'vs' enumerate In-Reply-To: <001801c1ef13$1d97f6c0$85e77ad1@othello> References: <000901c1eec0$410a2540$7cec7ad1@othello> <200204282214.g3SMEPI17614@pcp742651pcs.reston01.va.comcast.net> <001801c1ef13$1d97f6c0$85e77ad1@othello> Message-ID: <02042907590500.25806@arthur> On Monday 29 April 2002 02:16, Raymond Hettinger wrote: ... > I posted this one separately because zip() eats memory like crazy > and because a Python generator version crawls like a snail. > > IMHO, This is a better way to loop over multiple sequences and > has a chance at becoming the tool of choice. I scanned all of my Just musing -- what should enumerate() do when called with N!=1 arguments? Right now it raises, maybe not the most useful thing. Shouldn't it return an iterator for tuples of N+1 items with zip-like behavior? That would neatly substitute for some of my uses of zip, main use cases left out being of the form dict(zip(s1,s2)). [And enumerate() called without arguments should return a "non terminating" iterator yielding 0, 1, 2, ... -- I have a lot of uses of xrange(sys.maxint) that have no real reason to stop at maxint, and other ways to build such an iter, which could go away. But this has little to do with iterzip]. This wouldn't replace all need for iterzip (but might in many cases) but in any case it would be a natural and useful generalization of enumerate's design. YAGNI doesn't seem to apply since looking at my own code with such a general enumerate in mind does show reasonable need (admittedly I crunch a lot of integers in various combinatorial ways and just love the iterator/unending-stream computing architecture/paradigm). > > In general I'm not keen on increasing the number of builtin functions > > much. > > Ditto. Any chance of moving functions like map(), reduce(), and filter() > to a functional module; pow() and divmod() to the math module; or > input() to oblivion? The amount of code breakage would be appalling unless some easy way was provided for the programmer to assert "this (module/script/ package/entire Python run) needs the following well-known (names/ name sets) to be installed as built-in names", defaulting at least for now to functional.{math,reduce,filter}, etc. Quite apart from the math.pow issue (current builtin pow should need to go elsewhere than in math -- maybe a new module for "integral-oriented" arithmetic). site and optionally sitecustomize could perhaps help control such a hypothetical "easy way". Easiest in a sense would be do to it directly with import __builtin__ &c, but if that's to remain "strongly discouraged" then some fancy-dress for it, perhaps "unavailable after startup" in the same vein as sys.defaultencoding, could be provided. Perhaps too kludgey, but otherwise it seems to me that no removals from __builtin__ can happen before mythical Python-3000. Alex From bernie@3captus.com Mon Apr 29 07:14:44 2002 From: bernie@3captus.com (Bernard Yue) Date: Mon, 29 Apr 2002 00:14:44 -0600 Subject: [Python-Dev] timeoutsocket patch References: <3CCB891A.4E60C493@3captus.com> <20020429003413.B12835@eecs.tufts.edu> Message-ID: <3CCCE4D4.C005AA7F@3captus.com> Michael Gilfix wrote: > > Hmm, how do you poll sockets on OS/2 then? If there is some way, > then maybe that hack should go into the select library (the select > library treats sockets specially as polleable objects). Perhaps I > should rework the patch so that use of select is avoided if either the > import fails or the timeout is None. But (and maybe Guido will offer > his opinion here), I'd like to hold out to see if it really is the > case that some system doesn't support a select equivalent for sockets. > > -- Mike > Just found http://www.ia.amu.edu.pl/edm2/0401/socket3.html on the web. The documentation above mentioned select() support in IBM TCP/IP development toolkit for OS/2. So maybe OS/2 support select(). I remember TCP/IP support for OS/2 comes as an optional package of IBM LAN Manager client, back in 94-95. Am I correct, Andrew? Anyway, since you have access to OS/2, can you help us to dig in a bit? Bernie > On Mon, Apr 29 @ 08:12, Andrew MacIntyre wrote: > > Thanks. The EMX runtime library supports select() on sockets; I don't > > believe that native OS/2 has select() at all, but can't confirm. > > -- > Michael Gilfix > mgilfix@eecs.tufts.edu > > For my gpg public key: > http://www.eecs.tufts.edu/~mgilfix/contact.html -- There are three schools of magic. One: State a tautology, then ring the changes on its corollaries; that's philosophy. Two: Record many facts. Try to find a pattern. Then make a wrong guess at the next fact; that's science. Three: Be aware that you live in a malevolent Universe controlled by Murphy's Law, sometimes offset by Brewster's Factor; that's engineering. -- Robert A. Heinlein From guido@python.org Mon Apr 29 13:04:45 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 29 Apr 2002 08:04:45 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Your message of "Sun, 28 Apr 2002 21:05:42 EDT." <15564.40038.610923.30406@anthem.wooz.org> References: <000901c1eec0$410a2540$7cec7ad1@othello> <200204282214.g3SMEPI17614@pcp742651pcs.reston01.va.comcast.net> <001801c1ef13$1d97f6c0$85e77ad1@othello> <15564.40038.610923.30406@anthem.wooz.org> Message-ID: <200204291204.g3TC4kE19094@pcp742651pcs.reston01.va.comcast.net> [Barry] > I'd much rather see a patch that just changed zip() to be an iterator > instead of adding an iterzip(). I doubt that much in-field code would > break because of it (but write the PEP to find out. ;). You're brave. Changing something incompatibly that's been in the language since 2.1? I wish you luck though -- zip() really should have been an interator. > GvR> I wish. Since they were there first, it's hard to get rid of > GvR> them. (If you're truly masochist, write a PEP and post it to > GvR> c.l.py to find out how hard. :-) > > The PEP would have to specify a migration plan, i.e. the builtins are > identical to the functional module versions, and would a deprecation > schedule, etc. I wasn't serious. There's no way we can deprecate any builtin before Python 3. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 29 13:18:52 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 29 Apr 2002 08:18:52 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Your message of "Sun, 28 Apr 2002 21:39:53 EDT." <000b01c1ef1e$c39ba7a0$47e97ad1@othello> References: <000901c1eec0$410a2540$7cec7ad1@othello><200204282214.g3SMEPI17614@pcp742651pcs.reston01.va.comcast.net><001801c1ef13$1d97f6c0$85e77ad1@othello> <15564.40038.610923.30406@anthem.wooz.org> <000b01c1ef1e$c39ba7a0$47e97ad1@othello> Message-ID: <200204291218.g3TCIqY19143@pcp742651pcs.reston01.va.comcast.net> > Will do. I think the resistance to moving the functionals will be > fierce. You might try to deprecate reduce() though. It's useless. > Divmod probably has very few instances in real code. Not so sure. It's very handy in converting numbers to weird-radix or mixed-radix systems. E.g. it's a natural for converting posix timestamps to broken-out times, and e.g. /nondist/sandbox/datetime/datetime.py is full of it. > I think __pow__ would need to be left in (as the ** that calls it), > but the function itself may be used only rarely. Unfortunately the function is needed as the API to 3-arg pow() -- which doesn't easily fit in the math library since it's not a floating point thing. > Does anyone know of an automated what that I can scan a large body > of published Python code. I would want some real usage statistics > in a PEP but hate pulling modules down one at a time and grepping > them. You can write something using urllib that pulls things down, and I recommend using the tokenizer module to do the parsing -- much slower than grep, but doesn't get confused by comments etc. See Tools/scripts/finddiv.py for an example. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 29 13:30:55 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 29 Apr 2002 08:30:55 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Your message of "Sun, 28 Apr 2002 23:04:04 EDT." <009001c1ef2a$86896260$47e97ad1@othello> References: <000901c1eec0$410a2540$7cec7ad1@othello> <200204282214.g3SMEPI17614@pcp742651pcs.reston01.va.comcast.net> <001801c1ef13$1d97f6c0$85e77ad1@othello> <200204290032.g3T0WrU18240@pcp742651pcs.reston01.va.comcast.net> <009001c1ef2a$86896260$47e97ad1@othello> Message-ID: <200204291230.g3TCUt719180@pcp742651pcs.reston01.va.comcast.net> > > Did you time any of these? > > I just timed it and am shocked. iterzip() has exactly the same code > as zip() except for the final append result to list. So, I expected only > a microscopic speed-up. I don't see where the huge performance > improvement came from. Probably the list-append still has a slight quadratic component -- you're doing a million elements here. But I asked something else. How much does the speed difference affect the apps you have? In my experience it's usually used for small lists where the quadratic effect doesn't occur and the timing doesn't matter compared to the rest of the program. --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Mon Apr 29 14:30:42 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 29 Apr 2002 09:30:42 -0400 Subject: [Python-Dev] Re: iterzip() Message-ID: <15565.19202.571318.879377@anthem.wooz.org> [Barry] > I'd much rather see a patch that just changed zip() to be an iterator > instead of adding an iterzip(). I doubt that much in-field code would > break because of it (but write the PEP to find out. ;). Gvr> You're brave. Changing something incompatibly that's been in Gvr> the language since 2.1? I wish you luck though -- zip() Gvr> really should have been an interator. My point was that since zip()'s output seems to be[*] used mostly in for-loop contexts, the incompatibility is mostly theoretical. I could be wrong, or have too small a sample. This doesn't imply that I actually think it's worth changing, I don't. ;) I think most of the lists operated on by zip() are small enough to fit comfortably in memory. -Barry [*] Quickly audited: Python library, site-packages, Zope3, Mailman. All but one (not including test suite) were used as the sequence source in a for-loop or list comp. One was as an argument to list.extend(). From just@letterror.com Mon Apr 29 16:15:11 2002 From: just@letterror.com (Just van Rossum) Date: Mon, 29 Apr 2002 17:15:11 +0200 Subject: [Python-Dev] Re: iterzip() In-Reply-To: <15565.19202.571318.879377@anthem.wooz.org> Message-ID: Barry A. Warsaw wrote: > My point was that since zip()'s output seems to be[*] used mostly in > for-loop contexts, the incompatibility is mostly theoretical. The very first example I found of zip() in my own code does a sort() on the result, in a Schwarzian-transform-like sort method. It's pretty useful that way, so I doubt my own code is unique with this usage. I totally agree that zip() should have been an iterator, the same could be said of range()... It's simply too late. Just From trentm@ActiveState.com Mon Apr 29 17:54:33 2002 From: trentm@ActiveState.com (Trent Mick) Date: Mon, 29 Apr 2002 09:54:33 -0700 Subject: [Python-Dev] iterzip() In-Reply-To: <15564.42536.646377.149285@anthem.wooz.org>; from barry@zope.com on Sun, Apr 28, 2002 at 09:47:20PM -0400 References: <000901c1eec0$410a2540$7cec7ad1@othello> <200204282214.g3SMEPI17614@pcp742651pcs.reston01.va.comcast.net> <001801c1ef13$1d97f6c0$85e77ad1@othello> <15564.40038.610923.30406@anthem.wooz.org> <000b01c1ef1e$c39ba7a0$47e97ad1@othello> <15564.42536.646377.149285@anthem.wooz.org> Message-ID: <20020429095433.A28636@ActiveState.com> [Barry Warsaw wrote] > RH> Does anyone know of an automated what that I can scan a large > RH> body of published Python code. I would want some real usage > RH> statistics in a PEP but hate pulling modules down one at a > RH> time and grepping them. > > No, but it would be a really cool tool. And if it's available via cgi > or some other commonly available process, Mark Hammond could add it to > his neat Mozilla sidebar. Another thought: setup an LXR for Python's source and another LXR for a bunch of common Python extensions/modules. Trent -- Trent Mick TrentM@ActiveState.com From pinard@iro.umontreal.ca Mon Apr 29 18:07:43 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 29 Apr 2002 13:07:43 -0400 Subject: [Python-Dev] Re: [development doc updates] In-Reply-To: <15532.63699.122156.945015@grendel.zope.com> References: <20020404225927.530B318EAD1@grendel.zope.com> <20020404174752.M16962@tummy.com> <15532.63699.122156.945015@grendel.zope.com> Message-ID: [Fred L. Drake, Jr.] > The GNU info files have been contributed, but a patch has just been > submitted that I'll try to get to in time. In the past, we've run > into problems with elisp library incompatibilies between emacs and > xemacs. We'll see how the proposed patch holds up. If the Emacs Lisp incompatibilities come from the Emacs/XEmacs Texinfo formatting modules, I would guess you might be better off using the `makeinfo' tool instead, which is much more speedy, anyway. `makeinfo' (as well as the stand-alone `info' reader) are part of the Texinfo distribution release. I did not closely follow recent evolution in that area, however, so please take my comment with a grain of salt. -- François Pinard http://www.iro.umontreal.ca/~pinard From python@rcn.com Mon Apr 29 18:10:41 2002 From: python@rcn.com (Raymond Hettinger) Date: Mon, 29 Apr 2002 13:10:41 -0400 Subject: [Python-Dev] iterzip() References: <000901c1eec0$410a2540$7cec7ad1@othello> <200204282214.g3SMEPI17614@pcp742651pcs.reston01.va.comcast.net> <001801c1ef13$1d97f6c0$85e77ad1@othello> <200204290032.g3T0WrU18240@pcp742651pcs.reston01.va.comcast.net> <009001c1ef2a$86896260$47e97ad1@othello> <200204291230.g3TCUt719180@pcp742651pcs.reston01.va.comcast.net> Message-ID: <000b01c1efa0$df800740$95d8accf@othello> From: "Guido van Rossum" > > > Did you time any of these? > > > > I just timed it and am shocked. iterzip() has exactly the same code > > as zip() except for the final append result to list. So, I expected only > > a microscopic speed-up. I don't see where the huge performance > > improvement came from. > > Probably the list-append still has a slight quadratic component -- > you're doing a million elements here. The order of magnitude speed boost is consistent through all ranges measurable on my machine: n time func - ---- ---- 10000 0.11 zip 10000 0.00 iterzip 10000 0.38 genzip 100000 1.05 zip 100000 0.16 iterzip 100000 2.31 genzip 1000000 19.77 zip 1000000 1.37 iterzip 1000000 28.62 genzip And at each range, timing('list(iterzip)') is approx equal to timing('zip'). > But I asked something else. How much does the speed difference affect > the apps you have? In my experience it's usually used for small lists > where the quadratic effect doesn't occur and the timing doesn't matter I use zip() a lot, but iterzip() only made a speed difference in a few places: building dictionaries: reachable = dict(zip(map(alpha,wordlist,), wordlist)) forming character pairs on a file read into a string: zip(text[:-1], text[1:]) forming matrices from vectors: Mat( zip(xvec, yvec) ) If we have to leave zip untouched, consider adding iterzip anyway. It will almost always be the better choice. The real issue is which ONE of the following poisons are you most willing to swallow: 1. Forgo iterzip's speed/space improvements 2. Grow the number of builtins by one 3. Deprecate and/or relocate one of: zip, reduce, input, apply, oct 4. Change the behavior of zip to return an iterator (breaking some code) I vote for either #2 or #3. Raymond Hettinger P.S. Hmm, no name controversy? Alex hasn't even suggested a beautiful sounding Italian variant like 'terziparare' From tim.one@comcast.net Mon Apr 29 18:43:42 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 29 Apr 2002 13:43:42 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <200204291230.g3TCUt719180@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > Probably the list-append still has a slight quadratic component -- > you're doing a million elements here. I get: justpush 1.39 justzip 7.87 from this: """ from time import clock as now n = 1000000 zeroes = [0] * n def justpush(): x = [] push = x.append for i in zeroes: push(i) def justzip(): zip(zeroes) def run(func): start = now() func() finish = now() print "%10s %6.2f" % (func.__name__, finish - start) run(justpush) run(justzip) """ That is, at first glance it's *much* faster to do a million appends in pure Python than it is to let zip do them at C speed. I'm curious about what people get for this program on other platforms (pymalloc on or off may make a dramatic difference too -- or not, depending on how the platform malloc works). The cause on Win98SE is clear enough with a little instrumentation: in the justzip case, listobject.c's ins1 ends up *copying* the memory much more often, presumably due to that there's a new tuple allocation after every append (preventing realloc from growing into that space). justpush copies the memory at sizes (skipping some tiny ones, and where this is a count of the number of elements in the list, not the number of bytes (multiply by 4 to get the latter)): 247 639 959 2047 45055 163839 and that's all. justzip copies the memory at sizes: 247 639 959 2047 45055 53247 57343 65535 81919 98303 122879 163839 196607 229375 262143 294911 360447 393215 458751 589823 622591 688127 720895 753663 786431 819199 851967 884735 917503 950271 Copying 3-4MB multiple times at the end ain't cheap. justpush does relatively worse if the order in which they're called is swapped, as then justzip and the custom tuple free list conspire to leave memory in a more-fragmented state. > But I asked something else. How much does the speed difference affect > the apps you have? In my experience it's usually used for small lists > where the quadratic effect doesn't occur and the timing doesn't matter > compared to the rest of the program. Same here. High volume "vectorized" operations are NumPy's proper domain. From nas@python.ca Mon Apr 29 19:05:54 2002 From: nas@python.ca (Neil Schemenauer) Date: Mon, 29 Apr 2002 11:05:54 -0700 Subject: [Python-Dev] iterzip() In-Reply-To: ; from tim.one@comcast.net on Mon, Apr 29, 2002 at 01:43:42PM -0400 References: <200204291230.g3TCUt719180@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020429110554.A20248@glacier.arctrix.com> Tim Peters wrote: > I'm curious about what people get for this program on other platforms > (pymalloc on or off may make a dramatic difference too -- or not, > depending on how the platform malloc works). With pymalloc on Linux: justpush 0.79 justzip 5.17 From fredrik@pythonware.com Mon Apr 29 19:03:54 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Mon, 29 Apr 2002 20:03:54 +0200 Subject: [Python-Dev] iterzip() References: <000901c1eec0$410a2540$7cec7ad1@othello> <200204282214.g3SMEPI17614@pcp742651pcs.reston01.va.comcast.net> <001801c1ef13$1d97f6c0$85e77ad1@othello> <200204290032.g3T0WrU18240@pcp742651pcs.reston01.va.comcast.net> <009001c1ef2a$86896260$47e97ad1@othello> <200204291230.g3TCUt719180@pcp742651pcs.reston01.va.comcast.net> <000b01c1efa0$df800740$95d8accf@othello> Message-ID: <00a201c1efa8$481bcc10$ced241d5@hagrid> raymond wrote: > The real issue is which ONE of the following poisons are you most willing to > swallow: > 1. Forgo iterzip's speed/space improvements > 2. Grow the number of builtins by one > 3. Deprecate and/or relocate one of: zip, reduce, input, apply, oct > 4. Change the behavior of zip to return an iterator (breaking some code) > > I vote for either #2 or #3. here's item 5: 5. Put it in a suitable extension module (e.g. raymondTools), ship it outside python, let it be "out in the wild" for some suitable time, show that a considerable number of real users are using it in real life (and that you're writing real stuff yourself, and just not con- stantly arguing to change the language just because you can), and add it to the library if/when the time is right. I vote for #5. > 'terziparare' does that mean "false dilemma", or is it just a cool-sounding googlewhack? From guido@python.org Mon Apr 29 19:15:57 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 29 Apr 2002 14:15:57 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Your message of "Mon, 29 Apr 2002 13:10:41 EDT." <000b01c1efa0$df800740$95d8accf@othello> References: <000901c1eec0$410a2540$7cec7ad1@othello> <200204282214.g3SMEPI17614@pcp742651pcs.reston01.va.comcast.net> <001801c1ef13$1d97f6c0$85e77ad1@othello> <200204290032.g3T0WrU18240@pcp742651pcs.reston01.va.comcast.net> <009001c1ef2a$86896260$47e97ad1@othello> <200204291230.g3TCUt719180@pcp742651pcs.reston01.va.comcast.net> <000b01c1efa0$df800740$95d8accf@othello> Message-ID: <200204291815.g3TIFvM30566@pcp742651pcs.reston01.va.comcast.net> > The real issue is which ONE of the following poisons are you most willing to > swallow: > 1. Forgo iterzip's speed/space improvements > 2. Grow the number of builtins by one > 3. Deprecate and/or relocate one of: zip, reduce, input, apply, oct > 4. Change the behavior of zip to return an iterator (breaking some code) I strongly suggest sticking to the status quo. There are more important fish to fry. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Apr 29 19:22:25 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 29 Apr 2002 14:22:25 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Your message of "Mon, 29 Apr 2002 13:43:42 EDT." References: Message-ID: <200204291822.g3TIMPg30597@pcp742651pcs.reston01.va.comcast.net> > The cause on Win98SE is clear enough with a little instrumentation: in the > justzip case, listobject.c's ins1 ends up *copying* the memory much more > often, presumably due to that there's a new tuple allocation after every > append (preventing realloc from growing into that space). justpush copies > the memory at sizes (skipping some tiny ones, and where this is a count of > the number of elements in the list, not the number of bytes (multiply by 4 > to get the latter)): > > 247 639 959 2047 45055 163839 > > and that's all. justzip copies the memory at sizes: > > 247 639 959 2047 45055 53247 57343 65535 81919 98303 122879 163839 > 196607 229375 262143 294911 360447 393215 458751 589823 622591 > 688127 720895 753663 786431 819199 851967 884735 917503 950271 > > Copying 3-4MB multiple times at the end ain't cheap. > > justpush does relatively worse if the order in which they're called is > swapped, as then justzip and the custom tuple free list conspire to leave > memory in a more-fragmented state. This suggests that zip() can be sped up considerably by using a hint for the input sequence sizes. If they all respond properly to PySequence_Length(), zip can preallocate the result list to the correct size. It shouldn't believe the numbers fully, but using them as a hint for initial allocation makes a lot of sense, and should greatly reduce the inefficiency. --Guido van Rossum (home page: http://www.python.org/~guido/) From python@rcn.com Mon Apr 29 19:20:04 2002 From: python@rcn.com (Raymond Hettinger) Date: Mon, 29 Apr 2002 14:20:04 -0400 Subject: [Python-Dev] iterzip() References: <000901c1eec0$410a2540$7cec7ad1@othello> <200204282214.g3SMEPI17614@pcp742651pcs.reston01.va.comcast.net> <001801c1ef13$1d97f6c0$85e77ad1@othello> <200204290032.g3T0WrU18240@pcp742651pcs.reston01.va.comcast.net> <009001c1ef2a$86896260$47e97ad1@othello> <200204291230.g3TCUt719180@pcp742651pcs.reston01.va.comcast.net> <000b01c1efa0$df800740$95d8accf@othello> <200204291815.g3TIFvM30566@pcp742651pcs.reston01.va.comcast.net> Message-ID: <000c01c1efaa$7c6d4820$2366accf@othello> From: "Guido van Rossum" > I strongly suggest sticking to the status quo. There are more > important fish to fry. Okay. I'll go back to frying fish (working on the docs and an iterator tools module). If you don't mind, I'll leave the patch open in case the desire for it grows. Raymond Hettinger From tim.one@comcast.net Mon Apr 29 20:35:37 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 29 Apr 2002 15:35:37 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <200204291822.g3TIMPg30597@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > This suggests that zip() can be sped up considerably by using a hint > for the input sequence sizes. If they all respond properly to > PySequence_Length(), zip can preallocate the result list to the > correct size. It shouldn't believe the numbers fully, but using them > as a hint for initial allocation makes a lot of sense, and should > greatly reduce the inefficiency. I'm afraid I misanalyzed the cause. Giving zip a good guess about the result size and using PyList_SET_ITEM up until then leads (as expected) to no list copying in the test case I posted, but only dropped the runtime from 7.87 to 7.31 seconds (which makes sense looking at the numbers more closely: it doesn't take *that* long to copy a few dozen mebabytes of memory unless you're doing it by hand ). Check this out: """ from time import clock as now n = 1000000 def juststore(x): for i in x: x[i] = 0 def justtups(x): for i in x: 0, def storetups(x): for i in x: x[i] = 0, def run(func): x = range(n) start = now() func(x) finish = now() print "%10s %6.2f" % (func.__name__, finish - start) run(juststore) run(justtups) run(storetups) """ I get: juststore 0.93 justtups 0.58 storetups 7.61 list.append is out of the picture here. Creating a million tuples goes fast so long as they're recycled, and storing a million things goes fast, but storing a million distinct tuples takes very much longer. It's a Mystery to me so far. How does it act on Linux? From guido@python.org Mon Apr 29 20:55:15 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 29 Apr 2002 15:55:15 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Your message of "Mon, 29 Apr 2002 15:35:37 EDT." References: Message-ID: <200204291955.g3TJtFs31008@pcp742651pcs.reston01.va.comcast.net> > I get: > > juststore 0.93 > justtups 0.58 > storetups 7.61 > > list.append is out of the picture here. Creating a million tuples > goes fast so long as they're recycled, and storing a million things > goes fast, but storing a million distinct tuples takes very much > longer. It's a Mystery to me so far. How does it act on Linux? Very reasonable with Python 2.2: juststore 0.95 justtups 0.82 storetups 1.61 But with Python 2.3 it does just about as poorly as you saw on Windows: juststore 0.91 justtups 0.82 storetups 9.40 Could it be that pymalloc somehow slows things down here? The 2.3 tupleobject.c is nearly unchanged from 2.2, so that can't be it. If I double the array size, the 2.3 storetups time almost quadruples: juststore 1.82 justtups 1.62 storetups 34.11 While with 2.2 it's nicely linear: juststore 1.91 justtups 1.63 storetups 3.20 Bizarre, but I know you like a challenge. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Mon Apr 29 20:57:39 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 29 Apr 2002 15:57:39 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Message-ID: [Tim, on def juststore(x): for i in x: x[i] = 0 def justtups(x): for i in x: 0, def storetups(x): for i in x: x[i] = 0, where x is always range(1000000)] > ... > juststore 0.93 > justtups 0.58 > storetups 7.61 > > ... > It's a Mystery to me so far. How does it act on Linux? Well, it's not pymalloc here. pymalloc is happier recycling memory than grabbing arena after arena of persistent memory (it doesn't do what you think here -- it creates a pool's free list "lazily", not all at once, and that takes time). If I recompile with pymalloc disabled, the storetups time increases to 9.65 seconds. So despite pymalloc's reluctance to carve up memory until absolutely needed, it's still a lot faster than using the system malloc for small tuples on this box. From jeremy@zope.com Mon Apr 29 21:03:48 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 29 Apr 2002 16:03:48 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: References: <200204291822.g3TIMPg30597@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15565.42788.576934.21293@slothrop.zope.com> >>>>> "TP" == Tim Peters writes: TP> Check this out: TP> """ from time import clock as now TP> n = 1000000 TP> def juststore(x): TP> for i in x: x[i] = 0 TP> def justtups(x): TP> for i in x: 0, TP> def storetups(x): TP> for i in x: x[i] = 0, TP> def run(func): TP> x = range(n) start = now() func(x) finish = now() print TP> "%10s %6.2f" % (func.__name__, finish - start) TP> run(juststore) run(justtups) run(storetups) """ TP> I get: TP> juststore 0.93 TP> justtups 0.58 TP> storetups 7.61 TP> list.append is out of the picture here. Creating a million TP> tuples goes fast so long as they're recycled, and storing a TP> million things goes fast, but storing a million distinct tuples TP> takes very much longer. It's a Mystery to me so far. How does TP> it act on Linux? It acts about the same on Linux. I don't see why it's a mystery. justtups() only uses one tuple at a time; it gets re-used from the free list every time. storetups() has to (py)malloc a new tuple every stinkin' time. Note that there's a net excess of allocations in the storetup() case, so we invoke the garbage collector every n/700 times through the loop. I've noted before that it doesn't make much sense to invoke GC unless there is at least one deallocation; you can't reclaim anything if there are no DECREFs. Jeremy From pyth@devel.trillke.net Mon Apr 29 21:02:23 2002 From: pyth@devel.trillke.net (holger krekel) Date: Mon, 29 Apr 2002 22:02:23 +0200 Subject: [Python-Dev] iterzip() In-Reply-To: <200204291955.g3TJtFs31008@pcp742651pcs.reston01.va.comcast.net> References: <200204291955.g3TJtFs31008@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020429200223.GQ16217@devel.trillke> On Mon, Apr 29, 2002 at 03:55:15PM -0400, Guido van Rossum wrote: > > I get: > > > > juststore 0.93 > > justtups 0.58 > > storetups 7.61 > > > > list.append is out of the picture here. Creating a million tuples > > goes fast so long as they're recycled, and storing a million things > > goes fast, but storing a million distinct tuples takes very much > > longer. It's a Mystery to me so far. How does it act on Linux? > > Very reasonable with Python 2.2: > > juststore 0.95 > justtups 0.82 > storetups 1.61 Strange. my linux with python2.2.1rc2 (default) gives me: juststore 0.93 justtups 0.83 storetups 10.13 __holger From guido@python.org Mon Apr 29 21:10:36 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 29 Apr 2002 16:10:36 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Your message of "Mon, 29 Apr 2002 16:03:48 EDT." <15565.42788.576934.21293@slothrop.zope.com> References: <200204291822.g3TIMPg30597@pcp742651pcs.reston01.va.comcast.net> <15565.42788.576934.21293@slothrop.zope.com> Message-ID: <200204292010.g3TKAam31109@pcp742651pcs.reston01.va.comcast.net> > It acts about the same on Linux. > > I don't see why it's a mystery. justtups() only uses one tuple at a > time; it gets re-used from the free list every time. storetups() has > to (py)malloc a new tuple every stinkin' time. > > Note that there's a net excess of allocations in the storetup() case, > so we invoke the garbage collector every n/700 times through the > loop. I've noted before that it doesn't make much sense to invoke GC > unless there is at least one deallocation; you can't reclaim anything > if there are no DECREFs. But note the difference between 2.2 and 2.3 that I measured (repeatedly -- I don't know why Holger doesn't see this). Did the GC algorithm change between the two? *Something* must've changed. --Guido van Rossum (home page: http://www.python.org/~guido/) From nas@python.ca Mon Apr 29 21:13:20 2002 From: nas@python.ca (Neil Schemenauer) Date: Mon, 29 Apr 2002 13:13:20 -0700 Subject: [Python-Dev] iterzip() In-Reply-To: <15565.42788.576934.21293@slothrop.zope.com>; from jeremy@zope.com on Mon, Apr 29, 2002 at 04:03:48PM -0400 References: <200204291822.g3TIMPg30597@pcp742651pcs.reston01.va.comcast.net> <15565.42788.576934.21293@slothrop.zope.com> Message-ID: <20020429131320.A20713@glacier.arctrix.com> Jeremy Hylton wrote: > Note that there's a net excess of allocations in the storetup() case, > so we invoke the garbage collector every n/700 times through the > loop. That's the dirty culprit. :-) With the GC disabled: justpush 0.81 justzip 0.75 Perhaps we should raise the default threshold. Neil From jeremy@zope.com Mon Apr 29 21:16:56 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 29 Apr 2002 16:16:56 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <200204292010.g3TKAam31109@pcp742651pcs.reston01.va.comcast.net> References: <200204291822.g3TIMPg30597@pcp742651pcs.reston01.va.comcast.net> <15565.42788.576934.21293@slothrop.zope.com> <200204292010.g3TKAam31109@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15565.43576.659582.157078@slothrop.zope.com> I don't have a 2.2 checkout handy, but I've got 2.1 and 2.3. I don't see much difference. Note that I've made N smaller by about 10 to make it run in reasonable time on my (virtual) machine. localhost:~/src/python/src-release21-maint/build> ./python -O /tmp/iterzip.py juststore 0.41 justtups 0.40 storetups 2.77 localhost:~/src/python/src/build> ./python -O /tmp/iterzip.py juststore 0.32 justtups 0.29 storetups 2.44 Don't know what's up with your 2.2 build. Jeremy From tim.one@comcast.net Mon Apr 29 21:21:58 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 29 Apr 2002 16:21:58 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <15565.42788.576934.21293@slothrop.zope.com> Message-ID: [Jeremy] > It acts about the same on Linux. Also in 2.2.1? Guido presented some evidence there that's getting increasingly hard to believe (in part because holger krekel couldn't verify it, and in larger part because you found the true cause below ). > I don't see why it's a mystery. justtups() only uses one tuple at a > time; it gets re-used from the free list every time. storetups() has > to (py)malloc a new tuple every stinkin' time. It's a mystery because it's not credible that falling back to pymalloc is *that* much slower than using the tuple free list. > Note that there's a net excess of allocations in the storetup() case, > so we invoke the garbage collector every n/700 times through the > loop. Bingo! This also explains Guido's quadratic-time behavior. If I do gc.disable() at the start of this test, storetups() drops to under 1 second. So, also, the pymalloc burden is indeed comparatively trivial. > I've noted before that it doesn't make much sense to invoke GC > unless there is at least one deallocation; you can't reclaim anything > if there are no DECREFs. Let's see whether Neil can resist . From jeremy@zope.com Mon Apr 29 21:25:34 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 29 Apr 2002 16:25:34 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <20020429131320.A20713@glacier.arctrix.com> References: <200204291822.g3TIMPg30597@pcp742651pcs.reston01.va.comcast.net> <15565.42788.576934.21293@slothrop.zope.com> <20020429131320.A20713@glacier.arctrix.com> Message-ID: <15565.44094.375744.307284@slothrop.zope.com> >>>>> "NS" == Neil Schemenauer writes: NS> Jeremy Hylton wrote: >> Note that there's a net excess of allocations in the storetup() >> case, so we invoke the garbage collector every n/700 times >> through the loop. NS> That's the dirty culprit. :-) With the GC disabled: NS> justpush 0.81 NS> justzip 0.75 NS> Perhaps we should raise the default threshold. I suspect that would be good. Many Python programs chew through objects quickly. Still, for this class of programs, for any large N any small constant will be too small. If we care much about this class of examples (loops with no function calls and no deallocs), would it make sense to count alloc and dealloc separately? They would both need to exceed some limit before triggering GC. Jeremy From guido@python.org Mon Apr 29 21:25:18 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 29 Apr 2002 16:25:18 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Your message of "Mon, 29 Apr 2002 16:16:56 EDT." <15565.43576.659582.157078@slothrop.zope.com> References: <200204291822.g3TIMPg30597@pcp742651pcs.reston01.va.comcast.net> <15565.42788.576934.21293@slothrop.zope.com> <200204292010.g3TKAam31109@pcp742651pcs.reston01.va.comcast.net> <15565.43576.659582.157078@slothrop.zope.com> Message-ID: <200204292025.g3TKPJu05034@pcp742651pcs.reston01.va.comcast.net> > I don't have a 2.2 checkout handy, but I've got 2.1 and 2.3. I don't > see much difference. Note that I've made N smaller by about 10 to > make it run in reasonable time on my (virtual) machine. My 2.2 apparently had GC turned off. After a clean rebuild I see the same quadratic behavior too. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Mon Apr 29 21:56:43 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 29 Apr 2002 16:56:43 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <20020429131320.A20713@glacier.arctrix.com> Message-ID: [Neil Schemenauer] > That's the dirty culprit. :-) With the GC disabled: > > justpush 0.81 > justzip 0.75 > > Perhaps we should raise the default threshold. A fixed threshold of any size will leave us vulnerable to quadratic-time cases. Proportional growth wouldn't, though. For example, if a round of gc didn't find anything to collect, or found very little, we could boost the threshold by 25% (that's a right shift by 2 and an add ). Contrarily, when gc finds lots of stuff to collect, reduce the threshold. This adjusts itself to a program's runtime characteristics. I suspect most long-running programs enjoy vast stretches of time over which the second derivative of their gc behavior is relatively constant . From guido@python.org Mon Apr 29 22:07:23 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 29 Apr 2002 17:07:23 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Your message of "Mon, 29 Apr 2002 16:56:43 EDT." References: Message-ID: <200204292107.g3TL7NA05691@pcp742651pcs.reston01.va.comcast.net> > A fixed threshold of any size will leave us vulnerable to > quadratic-time cases. Proportional growth wouldn't, though. For > example, if a round of gc didn't find anything to collect, or found > very little, we could boost the threshold by 25% (that's a right > shift by 2 and an add ). Contrarily, when gc finds lots of > stuff to collect, reduce the threshold. This adjusts itself to a > program's runtime characteristics. I suspect most long-running > programs enjoy vast stretches of time over which the second > derivative of their gc behavior is relatively constant . Should we worry about programs that don't create any cyclical garbage for a long time, and then sudenly start creating lots of it? The initial GC-free period may bump the threshold up very far, and then it will build up a significant pile of cyclical garbage before GC runs again. --Guido van Rossum (home page: http://www.python.org/~guido/) From nas@python.ca Mon Apr 29 22:35:24 2002 From: nas@python.ca (Neil Schemenauer) Date: Mon, 29 Apr 2002 14:35:24 -0700 Subject: [Python-Dev] iterzip() In-Reply-To: <15565.42788.576934.21293@slothrop.zope.com>; from jeremy@zope.com on Mon, Apr 29, 2002 at 04:03:48PM -0400 References: <200204291822.g3TIMPg30597@pcp742651pcs.reston01.va.comcast.net> <15565.42788.576934.21293@slothrop.zope.com> Message-ID: <20020429143523.A20896@glacier.arctrix.com> Jeremy Hylton wrote: > I've noted before that it doesn't make much sense to invoke GC unless > there is at least one deallocation; you can't reclaim anything if > there are no DECREFs. It's easy to add a decref counter. It doesn't seem to help much though based on my limited testing. The standard test suite does not trigger it. I can't get our web application to either. Is it worth checking in? Maybe someone can come up with a better trick. Note that we must take into account generations as well. If threshold0 is low then there are lots of quick collections. Setting it larger does not help too much as new objects will eventually be examined (unless they are destroyed before the next collection). Neil From jeremy@zope.com Mon Apr 29 23:07:14 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 29 Apr 2002 18:07:14 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <20020429143523.A20896@glacier.arctrix.com> References: <200204291822.g3TIMPg30597@pcp742651pcs.reston01.va.comcast.net> <15565.42788.576934.21293@slothrop.zope.com> <20020429143523.A20896@glacier.arctrix.com> Message-ID: <15565.50194.536089.133382@slothrop.zope.com> >>>>> "NS" == Neil Schemenauer writes: NS> Jeremy Hylton wrote: >> I've noted before that it doesn't make much sense to invoke GC >> unless there is at least one deallocation; you can't reclaim >> anything if there are no DECREFs. NS> It's easy to add a decref counter. It doesn't seem to help much NS> though based on my limited testing. The standard test suite NS> does not trigger it. I can't get our web application to either. NS> Is it worth checking in? Maybe someone can come up with a NS> better trick. I'm not sure what your trick is, since you've only described it as a "decref counter." It may not be what I was thinking of. Regardless of whether it is, my idea may not be any good either. I was imagining a scheme like this: Count increfs and decrefs. Set two thresholds. A collection occurs when both thresholds are exceeded. Perhaps 100 decrefs and 1000 increfs. NS> Note that we must take into account generations as well. If NS> threshold0 is low then there are lots of quick collections. NS> Setting it larger does not help too much as new objects will NS> eventually be examined (unless they are destroyed before the NS> next collection). How does this come into play in the benchmark in question? It seems like we should have gotten a lot of quick collections, but it was still quite slow. Jeremy From guido@python.org Mon Apr 29 23:19:10 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 29 Apr 2002 18:19:10 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Your message of "Mon, 29 Apr 2002 18:07:14 EDT." <15565.50194.536089.133382@slothrop.zope.com> References: <200204291822.g3TIMPg30597@pcp742651pcs.reston01.va.comcast.net> <15565.42788.576934.21293@slothrop.zope.com> <20020429143523.A20896@glacier.arctrix.com> <15565.50194.536089.133382@slothrop.zope.com> Message-ID: <200204292219.g3TMJAX06078@pcp742651pcs.reston01.va.comcast.net> > I was imagining a scheme like this: Count increfs and decrefs. Set > two thresholds. A collection occurs when both thresholds are > exceeded. Perhaps 100 decrefs and 1000 increfs. I expect you can't literally count increfs and decrefs. These are macros that need to be super-fast, and I think we can't really afford to increment a counter on eacn macro invocation. The current thresholds are used to count the number of allocations. Adding the number of deallocations to mix seems dangerous: when (nearly) all data is tied up in cycles, there may not be any deallocations. It seems hard to distinguish between these two cases: (a) lots of memory is allocated and kept alive for real by containers (b) lots of memory is allocated and kept alive accidentally by cycles The zip example is a case of (a), but the same allocation behavior could ensue from case (b). Only running the allocator can determine which case we're seeing. I like Tim's idea of adjusting the thresholds base upon the effectiveness of recent collections. > How does this come into play in the benchmark in question? It seems > like we should have gotten a lot of quick collections, but it was > still quite slow. The benchmark has a list with a million elements, and during execution more and more tuples are added to it. I expect that somehow all these tuples are visited every 700 allocations, which gives quadratic behavior. I guess the generational part of the collector doesn't affect this -- the only way to reduce the work would be if the big list somehow was skipped once it was known to be "old". But since it contains references to "new" object (the 700 tuples allocated last), it probably ends up being visited anyway. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Mon Apr 29 23:17:37 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 29 Apr 2002 18:17:37 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <200204292107.g3TL7NA05691@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido, on adjusting gc threshold up or down depending on how much garbage each gc pass finds] > Should we worry about programs that don't create any cyclical garbage > for a long time, and then suddenly start creating lots of it? The > initial GC-free period may bump the threshold up very far, and then it > will build up a significant pile of cyclical garbage before GC runs > again. Yes, when the past is a rotten predictor of the future, it's hard to predict the future better than rottenly. Countless megabytes of cyclic structures may be kept alive by a single reference too, so looking for a bump in deallocation rate isn't a cure-all either. A practical compromise would be to allow thresholds to auto-adjust, but only within a fixed range. That would give Neil the opportunity to add more tuning knobs nobody understands . I'd set the default upper bound pretty high, like a few million excess allocations. Somebody creating oodles of cyclic trash after a long quiet period, *and* who can't afford to let the memory go unreclaimed, would have to learn to import gc and force the issue. The rest of us would enjoy less gc overhead with little potential for harm. I'd also do auto-adjust by more than 25%; *2 and /2 are more like it. Programs don't *ease* their way into different memory behaviors, when it changes it's likely to be a sharp break. From barry@zope.com Mon Apr 29 23:26:11 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 29 Apr 2002 18:26:11 -0400 Subject: [Python-Dev] iterzip() References: <200204291822.g3TIMPg30597@pcp742651pcs.reston01.va.comcast.net> <15565.42788.576934.21293@slothrop.zope.com> <20020429143523.A20896@glacier.arctrix.com> <15565.50194.536089.133382@slothrop.zope.com> <200204292219.g3TMJAX06078@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15565.51331.378027.783169@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> The current thresholds are used to count the number of GvR> allocations. Adding the number of deallocations to mix seems GvR> dangerous: when (nearly) all data is tied up in cycles, there GvR> may not be any deallocations. It seems hard to distinguish GvR> between these two cases: GvR> (a) lots of memory is allocated and kept alive for real by GvR> containers GvR> (b) lots of memory is allocated and kept alive accidentally GvR> by cycles GvR> The zip example is a case of (a), but the same allocation GvR> behavior could ensue from case (b). Only running the GvR> allocator can determine which case we're seeing. I like GvR> Tim's idea of adjusting the thresholds base upon the GvR> effectiveness of recent collections. Isn't this a case of "knowing your application"? IOW, you're doing something that the gc isn't well-tuned to handle, by default. That's why we expose the its operation through the gc module -- so you can take explicit steps for the hotspots in your application. Not to say we can't improve the tuning, but it'll never be perfect, so just try to make it good enough for the most common types of programs. Then document situations where it might not do so well. -Barry From jeremy@zope.com Mon Apr 29 23:31:57 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 29 Apr 2002 18:31:57 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <200204292219.g3TMJAX06078@pcp742651pcs.reston01.va.comcast.net> References: <200204291822.g3TIMPg30597@pcp742651pcs.reston01.va.comcast.net> <15565.42788.576934.21293@slothrop.zope.com> <20020429143523.A20896@glacier.arctrix.com> <15565.50194.536089.133382@slothrop.zope.com> <200204292219.g3TMJAX06078@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15565.51677.329491.714749@slothrop.zope.com> >>>>> "GvR" == Guido van Rossum writes: >> I was imagining a scheme like this: Count increfs and decrefs. >> Set two thresholds. A collection occurs when both thresholds are >> exceeded. Perhaps 100 decrefs and 1000 increfs. GvR> I expect you can't literally count increfs and decrefs. These GvR> are macros that need to be super-fast, and I think we can't GvR> really afford to increment a counter on eacn macro invocation. GvR> The current thresholds are used to count the number of GvR> allocations. I was being sloppy. I meant allocations and deallactions. GvR> Adding the number of deallocations to mix seems dangerous: when GvR> (nearly) all data is tied up in cycles, there may not be any GvR> deallocations. Probably right, although function calls should provide a steady stream of deallocations. Frame, locals, &c. are deallocated on exit. So unless the code is blocked waiting for those cycles to be collected, it ought to eventually make progress. GvR> It seems hard to distinguish between these two cases: GvR> (a) lots of memory is allocated and kept alive for real by GvR> containers GvR> (b) lots of memory is allocated and kept alive accidentally by GvR> cycles GvR> The zip example is a case of (a), but the same allocation GvR> behavior could ensue from case (b). Only running the allocator GvR> can determine which case we're seeing. I like Tim's idea of GvR> adjusting the thresholds base upon the effectiveness of recent GvR> collections. I agree that this sounds interesting. >> How does this come into play in the benchmark in question? It >> seems like we should have gotten a lot of quick collections, but >> it was still quite slow. GvR> The benchmark has a list with a million elements, and during GvR> execution more and more tuples are added to it. I expect that GvR> somehow all these tuples are visited every 700 allocations, GvR> which gives quadratic behavior. I guess the generational part GvR> of the collector doesn't affect this -- I guess this is a question for Neil. I assumed that the generational part would affect this. GvR> the only way to reduce the work would be if the big list GvR> somehow was skipped once it was known to be "old". But since GvR> it contains references to "new" object (the 700 tuples GvR> allocated last), it probably ends up being visited anyway. I thought something was labelled old after it survived some number of collections. Why would the age of the objects it contains have any affect? Jeremy From nas@python.ca Mon Apr 29 23:52:47 2002 From: nas@python.ca (Neil Schemenauer) Date: Mon, 29 Apr 2002 15:52:47 -0700 Subject: [Python-Dev] iterzip() In-Reply-To: <15565.50194.536089.133382@slothrop.zope.com>; from jeremy@zope.com on Mon, Apr 29, 2002 at 06:07:14PM -0400 References: <200204291822.g3TIMPg30597@pcp742651pcs.reston01.va.comcast.net> <15565.42788.576934.21293@slothrop.zope.com> <20020429143523.A20896@glacier.arctrix.com> <15565.50194.536089.133382@slothrop.zope.com> Message-ID: <20020429155247.B20896@glacier.arctrix.com> Jeremy Hylton wrote: > I'm not sure what your trick is, since you've only described it as a > "decref counter." Sorry. I'm keeping track of the number of calls to PyObject_GC_Del since the last collection. While it's zero, collection doesn't happen. That makes the justzip function run fast but doesn't seem to help anywhere else. > I was imagining a scheme like this: Count increfs and decrefs. Set > two thresholds. A collection occurs when both thresholds are > exceeded. Perhaps 100 decrefs and 1000 increfs. That would cut down on the collections more but I'm not sure how much in practice. In real code it seems like allocations and deallocations are pretty mixed up. > How does this come into play in the benchmark in question? It seems > like we should have gotten a lot of quick collections, but it was > still quite slow. The GC cost is paid early and the objects get put in an older generation. Obviously that's a waste of time if they are deallocated in the near future. justpush deallocates as it goes so the GC is never triggered. I just tried measuring the time spent in the GC while loading some nasty web pages in our system (stuff that looks at thousands of objects). I used the Pentium cycle counter since clock(2) seems to have very low resolution. Setting threshold0 to 7500 makes the GC take up twice the amount of time as with the default settings (700). That surprised me. I thought it wouldn't make much difference. Maybe I screwed up. :-) Neil From jeremy@zope.com Tue Apr 30 00:02:52 2002 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 29 Apr 2002 19:02:52 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <20020429155247.B20896@glacier.arctrix.com> References: <200204291822.g3TIMPg30597@pcp742651pcs.reston01.va.comcast.net> <15565.42788.576934.21293@slothrop.zope.com> <20020429143523.A20896@glacier.arctrix.com> <15565.50194.536089.133382@slothrop.zope.com> <20020429155247.B20896@glacier.arctrix.com> Message-ID: <15565.53532.179449.273012@slothrop.zope.com> >>>>> "NS" == Neil Schemenauer writes: >> How does this come into play in the benchmark in question? It >> seems like we should have gotten a lot of quick collections, but >> it was still quite slow. NS> The GC cost is paid early and the objects get put in an older NS> generation. Obviously that's a waste of time if they are NS> deallocated in the near future. justpush deallocates as it goes NS> so the GC is never triggered. The "0," tuples aren't deallocated until the end of the function, so it seems good to get them out of the current generation. NS> I just tried measuring the time spent in the GC while loading NS> some nasty web pages in our system (stuff that looks at NS> thousands of objects). I used the Pentium cycle counter since NS> clock(2) seems to have very low resolution. Setting threshold0 NS> to 7500 makes the GC take up twice the amount of time as with NS> the default settings (700). That surprised me. I thought it NS> wouldn't make much difference. Maybe I screwed up. :-) Here's some data from gprof to be puzzled by. I ran Tim's test with only run(storetups) enabled. Here's the top of the output: Each sample counts as 0.01 seconds. % cumulative self self total time seconds seconds calls ms/call ms/call name 60.19 0.62 0.62 112233 0.01 0.01 PyObject_Malloc 6.80 0.69 0.07 622804 0.00 0.00 visit_move 4.85 0.74 0.05 621301 0.00 0.00 visit_decref 4.85 0.79 0.05 168 0.30 0.67 list_traverse 3.88 0.83 0.04 145 0.28 0.41 move_roots 2.91 0.86 0.03 567141 0.00 0.00 tupletraverse 1.94 0.88 0.02 286727 0.00 0.00 gc_list_append 1.94 0.90 0.02 102050 0.00 0.00 tupledealloc 1.94 0.92 0.02 145 0.14 0.89 move_root_reachable 0.97 0.93 0.01 286727 0.00 0.00 gc_list_remove 0.97 0.94 0.01 113171 0.00 0.00 PyObject_Free 0.97 0.95 0.01 100043 0.00 0.00 list_ass_item 0.97 0.96 0.01 32702 0.00 0.00 PyDict_Next 0.97 0.97 0.01 489 0.02 1.78 eval_frame 0.97 0.98 0.01 280 0.04 0.04 instancemethod_dealloc 0.97 0.99 0.01 145 0.07 0.82 subtract_refs 0.97 1.00 0.01 145 0.07 0.07 update_refs 0.97 1.01 0.01 7 1.43 1.43 posix_stat 0.97 1.02 0.01 6 1.67 1.67 member_get 0.97 1.03 0.01 1 10.00 10.00 PyInt_Fini So the profile output suggests that it's spending 60% of its time in pymalloc. Jeremy From guido@python.org Tue Apr 30 00:01:52 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 29 Apr 2002 19:01:52 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Your message of "Mon, 29 Apr 2002 18:31:57 EDT." <15565.51677.329491.714749@slothrop.zope.com> References: <200204291822.g3TIMPg30597@pcp742651pcs.reston01.va.comcast.net> <15565.42788.576934.21293@slothrop.zope.com> <20020429143523.A20896@glacier.arctrix.com> <15565.50194.536089.133382@slothrop.zope.com> <200204292219.g3TMJAX06078@pcp742651pcs.reston01.va.comcast.net> <15565.51677.329491.714749@slothrop.zope.com> Message-ID: <200204292301.g3TN1qr06155@pcp742651pcs.reston01.va.comcast.net> > I thought something was labelled old after it survived some number of > collections. Why would the age of the objects it contains have any > affect? I thought that old objects were traced less frequently to reduce the amount of time wasted on them. But maybe I don't understand the code. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Tue Apr 30 00:18:52 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 29 Apr 2002 19:18:52 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <15565.51677.329491.714749@slothrop.zope.com> Message-ID: Let me channel Neil here. "The list" does move out of the "frequently collected" generation (gen0) the first time that gets collected, and works its way into the least frequently collected generation (gen2). The time for futile attempts to collect the first- and second-most (gen1) frequently collected generations doesn't grow as the program goes on. After things get going, they just contain tuples, and those move from gen0 into gen1 into gen2 at the same rate they're created. The quadratic time is due to gen2 collections: gen2 holds "the list", and an ever-increasing number of tuples. gen0 never holds more than about 700 tuples, and gen1 never more than about 8500. This is easy to see by adding import gc gc.set_debug(gc.DEBUG_STATS) at the top of the test case. Here's a snippet from near the end of a run, taken at a time when gen1 and gen2 collections are about to trigger: ... gc: collecting generation 0... gc: objects in each generation: 701 4907 736679 gc: done. gc: collecting generation 0... gc: objects in each generation: 701 5608 736679 gc: done. gc: collecting generation 0... gc: objects in each generation: 701 6309 736679 gc: done. gc: collecting generation 0... gc: objects in each generation: 701 7010 736679 gc: done. gc: collecting generation 1... gc: objects in each generation: 0 8412 736679 gc: done. gc: collecting generation 2... gc: objects in each generation: 0 0 745792 ... If you try this, you'll get a good gut feel for what's happening: the output whizzes by until hitting a gen2 collection, then there's a pause, and the gen2 pause gets longer and longer as the program goes on. So the primary benefit in bad cases from auto-adjusting would come from auto-adjusting the gen2 threshold. That's the only generation that can grow without bound. Before an object moves into gen2, it can get futilely scanned just twice (once in a gen0 collection, and again in a gen1 collection). Boosting the gen0 threshold wouldn't have much effect. From tim.one@comcast.net Tue Apr 30 00:24:17 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 29 Apr 2002 19:24:17 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <200204292301.g3TN1qr06155@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > I thought that old objects were traced less frequently to reduce the > amount of time wasted on them. They are, but the set of oldest objects grows without bound in this case, and they still get traced from time to time. That's enough to yield quadratic-time behavior. > But maybe I don't understand the code. That too . From tim.one@comcast.net Tue Apr 30 00:42:22 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 29 Apr 2002 19:42:22 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <20020429155247.B20896@glacier.arctrix.com> Message-ID: [Neil Schemenauer] > ... > I just tried measuring the time spent in the GC while loading some nasty > web pages in our system (stuff that looks at thousands of objects). I > used the Pentium cycle counter since clock(2) seems to have very low > resolution. Setting threshold0 to 7500 makes the GC take up twice the > amount of time as with the default settings (700). That surprised me. > I thought it wouldn't make much difference. Maybe I screwed up. :-) Did you try reducing theshold0 too? Maybe GC time tends toward zero as threshold0 does . I wouldn't discount cache effects here. 7500 objects adds up to lot of bytes, and gc traversal touches stuff all over them. This is one good reason for why a gen0 traversal of the 7500 most-recently allocated still-live objects may take significantly longer than 10 gen0 traversals of the 750 suchlike. From tim.one@comcast.net Tue Apr 30 00:36:10 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 29 Apr 2002 19:36:10 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <15565.51331.378027.783169@anthem.wooz.org> Message-ID: [Barry] > Isn't this a case of "knowing your application"? IOW, you're doing > something that the gc isn't well-tuned to handle, by default. That's > why we expose the its operation through the gc module -- so you can > take explicit steps for the hotspots in your application. The difficulty is with apps that grow a lot of long-lived containers that aren't trash and don't even contain cycles. There's no bound on how often they'll get crawled over looking for trash that ain't there, and the more of those you grow the longer it takes to look at them. When a gen2 collection doesn't find any trash, it should probably become less eager to try looking at the same stuff yet again. Adding more generations could have a similar good effect. Half of a shadow of an idea: at least in my code, it's common to have gazillions of tuples, and they almost always contain just strings and numbers. Since they're immutable, they'll never contain anything else. So they could get unlinked from cyclic gc entirely without ill effect (it's not possible that they could ever be in a cycle). Perhaps a gen2 collection could learn something about this and automagically untrack them. From tim.one@comcast.net Tue Apr 30 00:51:27 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 29 Apr 2002 19:51:27 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <15565.53532.179449.273012@slothrop.zope.com> Message-ID: [Jeremy] > The "0," tuples aren't deallocated until the end of the function, so > it seems good to get them out of the current generation. They move out of gen0 and gen1 regularly and quickly; it's the gen2 collections that consume the time. > ... > Here's some data from gprof to be puzzled by. I ran Tim's test with > only run(storetups) enabled. Here's the top of the output: I think you also cut the size of the test by about a factor of 10, else you would have seen many more calls to PyObject_Malloc. > Each sample counts as 0.01 seconds. > % cumulative self self total > time seconds seconds calls ms/call ms/call name > 60.19 0.62 0.62 112233 0.01 0.01 PyObject_Malloc > .. > So the profile output suggests that it's spending 60% of its time in > pymalloc. Quite possible! If you cut the size of the test to 100,000 tuples, there's only one gen2 collection in the whole run, and it's gen2 collections that are the source of the problems. I expect you're mostly testing what a crappy job vmware does . From guido@python.org Tue Apr 30 00:56:36 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 29 Apr 2002 19:56:36 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Your message of "Mon, 29 Apr 2002 19:36:10 EDT." References: Message-ID: <200204292356.g3TNua306479@pcp742651pcs.reston01.va.comcast.net> > The difficulty is with apps that grow a lot of long-lived containers > that aren't trash and don't even contain cycles. There's no bound > on how often they'll get crawled over looking for trash that ain't > there, and the more of those you grow the longer it takes to look at > them. When a gen2 collection doesn't find any trash, it should > probably become less eager to try looking at the same stuff yet > again. Adding more generations could have a similar good effect. > > Half of a shadow of an idea: at least in my code, it's common to > have gazillions of tuples, and they almost always contain just > strings and numbers. Since they're immutable, they'll never contain > anything else. So they could get unlinked from cyclic gc entirely > without ill effect (it's not possible that they could ever be in a > cycle). Perhaps a gen2 collection could learn something about this > and automagically untrack them. Different (complementary) idea: how about having more generations, each being traversed less frequently than the previous one? Maybe a (potentially) infinite number of generations? (Or at least a fixed limit that never gets reached in practice.) Wouldn't this have the same effect as increasing the threshold exponentially? --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@comcast.net Tue Apr 30 01:02:09 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 29 Apr 2002 20:02:09 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <200204292356.g3TNua306479@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido] > Different (complementary) idea: how about having more generations, > each being traversed less frequently than the previous one? That's what I meant by "Adding more generations could have a similar good effect". It's certainly worth considering. > Maybe a (potentially) infinite number of generations? (Or at least a > fixed limit that never gets reached in practice.) Wouldn't this have > the same effect as increasing the threshold exponentially? Yes, but it wouldn't get the effect of decreasing the threshold exponentially when gc suddenly starts getting lots of trash back from the oldest generation. Only adaptive schemes actually adapt . From nas@python.ca Tue Apr 30 02:54:25 2002 From: nas@python.ca (Neil Schemenauer) Date: Mon, 29 Apr 2002 18:54:25 -0700 Subject: [Python-Dev] iterzip() In-Reply-To: ; from tim.one@comcast.net on Mon, Apr 29, 2002 at 08:02:09PM -0400 References: <200204292356.g3TNua306479@pcp742651pcs.reston01.va.comcast.net> Message-ID: <20020429185425.A21324@glacier.arctrix.com> Tim Peters wrote: > That's what I meant by "Adding more generations could have a similar good > effect". It's certainly worth considering. Adding a fourth generation drops the time from 5.13 to 2.11 on my machine. Adding a fifth doesn't seem to make a difference. I used 10 as the threshold for both new generations. Neil From tim.one@comcast.net Tue Apr 30 03:36:12 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 29 Apr 2002 22:36:12 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: <20020429185425.A21324@glacier.arctrix.com> Message-ID: [Neil Schemenauer] > Adding a fourth generation drops the time from 5.13 to 2.11 on my > machine. Adding a fifth doesn't seem to make a difference. I used 10 > as the threshold for both new generations. Alas, these thresholds are a little hard to work with. For example ... else if (collections0 > threshold1) { ... collections1++; /* merge gen0 into gen1 and collect gen1 */ ... collections0 = 0; } else { generation = 0; collections0++; ... /* collect gen0 */ ... } Let's say threshold1 is 10 (because it is ), and we just finished a gen1 collection. Then collections0 is 0. We have to do 11 gen0 collections then before "collections0 > threshold1" succeeds, and that point is actually the 12th time gen0 has filled up since the last time we did a gen1 collection. Similarly for collections1 vs threshold2. This makes it hard to multiply them out in an obvious way . Anyway, with 4 generations it takes in the ballpark of 700 * 10 * 10 * 10 = 700,000 excess allocations before a gen3 collection is triggered, so I expect you saw exactly one gen3 collection during the lifetime of the test run (there are about 1,000,000 excess allocations during its run). Also that adding a fifth generation wouldn't matter at all in this test, since you'd still see exactly one gen3 collection, and a gen4 collection would never happen. Now another ballpark: On the only machine that matters in real life (mine), I'm limited to 2**31 bytes of user address space, and an object participating in gc can rarely be smaller than 40 bytes. That means I can't have more than 2**31/40 ~= 55 million gcable objects alive at once, and that also bounds the aggregate excess of allocations over deallocations. That surprised me. It means the "one million tuple" test is already taxing a non-trivial percentage of this box's theoretical capacity. Indeed, I tried boosting it to 10 million, and after glorious endless minutes of listening to the disk grind itself to dust (with gc disabled, even), Win98 rebooted itself. So another factor-of-10 generation or two would probably move the gross surprises here out of the realm of practical concern. Except, of course, for the programs where it wouldn't . From aahz@pythoncraft.com Tue Apr 30 03:56:52 2002 From: aahz@pythoncraft.com (Aahz) Date: Mon, 29 Apr 2002 22:56:52 -0400 Subject: [Python-Dev] Billions of gc's In-Reply-To: <15565.51331.378027.783169@anthem.wooz.org> References: <200204291822.g3TIMPg30597@pcp742651pcs.reston01.va.comcast.net> <15565.42788.576934.21293@slothrop.zope.com> <20020429143523.A20896@glacier.arctrix.com> <15565.50194.536089.133382@slothrop.zope.com> <200204292219.g3TMJAX06078@pcp742651pcs.reston01.va.comcast.net> <15565.51331.378027.783169@anthem.wooz.org> Message-ID: <20020430025652.GA29745@panix.com> On Mon, Apr 29, 2002, Barry A. Warsaw wrote: > > Isn't this a case of "knowing your application"? IOW, you're doing > something that the gc isn't well-tuned to handle, by default. That's > why we expose the its operation through the gc module -- so you can > take explicit steps for the hotspots in your application. > > Not to say we can't improve the tuning, but it'll never be perfect, so > just try to make it good enough for the most common types of > programs. Then document situations where it might not do so well. My take is that programs with a million live objects and no cycles are common enough that gc should be designed to handle that smoothly. I don't think that a programmer casually writing such applications (say, processing information from a database) should be expected to understand gc well enough to tune it. Having read the entire discussion so far, and *NOT* being any kind of gc expert, I would say that Tim's adaptive solution makes the most sense to me. For years, we told people with cyclic data to figure out how to fix the problem themselves; now that we have gc available, I don't think we should punish everyone else. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "I used to have a .sig but I found it impossible to please everyone..." --SFJ From greg@cosc.canterbury.ac.nz Tue Apr 30 04:40:10 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 30 Apr 2002 15:40:10 +1200 (NZST) Subject: [Python-Dev] iterzip() In-Reply-To: <15564.40038.610923.30406@anthem.wooz.org> Message-ID: <200204300340.PAA02565@s454.cosc.canterbury.ac.nz> barry@zope.com (Barry A. Warsaw): > I'd much rather see a patch that just changed zip() to be an iterator > instead of adding an iterzip(). Wouldn't it be better as a lazy sequence a la xrange() that produces an iterator, rather than being an iterator itself? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Tue Apr 30 04:44:56 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 30 Apr 2002 15:44:56 +1200 (NZST) Subject: [Python-Dev] iterzip 'vs' enumerate In-Reply-To: <02042907590500.25806@arthur> Message-ID: <200204300344.PAA02569@s454.cosc.canterbury.ac.nz> Alex Martelli : > Just musing -- what should enumerate() do when called with N!=1 > arguments? Right now it raises, maybe not the most useful thing. > > Shouldn't it return an iterator for tuples of N+1 items with zip-like > behavior? Maybe instead of introducing a new function, we could just treat enumerate() as a variation on zip(), e.g. zip(seq, ..., numbered = 1) thereby sidestepping the naming problem altogether! Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim.one@comcast.net Tue Apr 30 04:55:41 2002 From: tim.one@comcast.net (Tim Peters) Date: Mon, 29 Apr 2002 23:55:41 -0400 Subject: [Python-Dev] Billions of gc's In-Reply-To: <20020430025652.GA29745@panix.com> Message-ID: [Aahz] > My take is that programs with a million live objects and no cycles are > common enough that gc should be designed to handle that smoothly. Well, millions of live objects is common but isn't a problem. The glitch we're looking at it is surprising slowdown with millions of live *container* objects. The latter isn't so common. > I don't think that a programmer casually writing such applications > (say, processing information from a database) should be expected to > understand gc well enough to tune it. People casually writing applications pushing the limits of their boxes are in for more surprises than just this . > Having read the entire discussion so far, and *NOT* being any kind of > gc expert, I would say that Tim's adaptive solution makes the most > sense to me. For years, we told people with cyclic data to figure out > how to fix the problem themselves; now that we have gc available, I > don't think we should punish everyone else. We're not trying to punish anyone, but innocent users with lots of containers can lose big despite our wishes: if we don't check them for cycles, they can run out of memory; if we do check them for cycles, it necessarily consumes time. As a datapoint, here are the times (in seconds) for justzip() on my box after my checkin to precompute the result size (list.append behavior is irrelevant now): gc disabled: 0.64 gc enabled: 7.32 magic=2(*): 2.63 magic=3(*): 2.02 (*) This is gcmodule.c fiddled to add this block after "collections1 = 0;" in the first branch of collect_generations(): if (n == 0) threshold2 *= magic; else if (threshold2 > 5) threshold2 /= magic; magic=1 is equivalent to the current code. That's all an "adaptive scheme" need amount to, provided the "*=" part were fiddled to prevent threshold2 from becoming insanely large. Boosting magic above 3 didn't do any more good in this test. At magic=3 it still takes 3+ times longer than with gc disabled, but that's a whale of a lot better than the current 11+ times longer. Note that with gc disabled, any cycle in any of the 1,000,001 containers this test creates would leak forever -- casual users definitely get something back for the time spent. From greg@cosc.canterbury.ac.nz Tue Apr 30 05:10:23 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 30 Apr 2002 16:10:23 +1200 (NZST) Subject: [Python-Dev] an oddball alternative name for enumerate() In-Reply-To: <200204261620.g3QGKZvU025640@email.nist.gov> Message-ID: <200204300410.QAA02580@s454.cosc.canterbury.ac.nz> Michael McLay : > Unfortunately the PEP has rejected all varations of the term index: > > All of the names involving 'index' clashed with usage in database > languages where indexing implies a sorting operation rather than > linear sequencing. I think that comment is badly worded. Creating a database index may involve sorting at some point, but that's irrelevant. It's called "indexing" by analogy with indexing a book -- i.e. creating a data structure allowing fast access to a data item given a key. I think this usage of the term is much more widely understood than just in the database field, or even the whole computer field. > I searched the Python document, using Thomas Heller's nifty "Index for > Python manuals". Why do you think that document has the word "Index" in its title... :-) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Tue Apr 30 05:13:53 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 30 Apr 2002 16:13:53 +1200 (NZST) Subject: [Python-Dev] Making None a keyword In-Reply-To: <200204261242.g3QCguQ02366@pcp742651pcs.reston01.va.comcast.net> Message-ID: <200204300413.QAA02583@s454.cosc.canterbury.ac.nz> Guido: > atom: '(' [testlist] ')' | '[' [listmaker] ']' | '{' [dictmaker] '}' > | '`' testlist '`' | NAME | NUMBER | STRING+ | 'None' > > It needs to add 'None' as an alternative to all other places where > currently NAME occurs. Wait a minute, you're going about this all wrong! Instead of making None a reserved word, why not just make it a non-reserved keyword when used as a variable name? Then you wouldn't have to change the grammar at all, and there would be no problem with the use of None in other contexts, such as attribute names. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From aahz@pythoncraft.com Tue Apr 30 05:21:26 2002 From: aahz@pythoncraft.com (Aahz) Date: Tue, 30 Apr 2002 00:21:26 -0400 Subject: [Python-Dev] Billions of gc's In-Reply-To: References: <20020430025652.GA29745@panix.com> Message-ID: <20020430042126.GA19504@panix.com> On Mon, Apr 29, 2002, Tim Peters wrote: > [Aahz] >> >> My take is that programs with a million live objects and no cycles are >> common enough that gc should be designed to handle that smoothly. > > Well, millions of live objects is common but isn't a problem. The glitch > we're looking at it is surprising slowdown with millions of live *container* > objects. The latter isn't so common. > >> I don't think that a programmer casually writing such applications >> (say, processing information from a database) should be expected to >> understand gc well enough to tune it. > > People casually writing applications pushing the limits of their boxes are > in for more surprises than just this . Fair enough. I hadn't quite understood that it was specifically container objects, but obviously a database result will have lots of tuples, so I think that's a good real-world metric for testing whatever solution is proposed. Here's a question: suppose we've got a database result with 10K rows (I'd say that is fairly common), and we're processing each row with a regex (something that can't be done in SQL). What's a ballpark for gc overhead before and after your fix? (I'm still not set up to compile CVS, so I can't do it myself.) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "I used to have a .sig but I found it impossible to please everyone..." --SFJ From tim.one@comcast.net Tue Apr 30 06:16:21 2002 From: tim.one@comcast.net (Tim Peters) Date: Tue, 30 Apr 2002 01:16:21 -0400 Subject: [Python-Dev] Billions of gc's In-Reply-To: <20020430042126.GA19504@panix.com> Message-ID: [Aahz] > ... > Here's a question: suppose we've got a database result with 10K rows > (I'd say that is fairly common), and we're processing each row with a > regex (something that can't be done in SQL). What's a ballpark for gc > overhead before and after your fix? (I'm still not set up to compile > CVS, so I can't do it myself.) What's a database? grateful-for-the-easy-ones-ly y'rs - tim From martin@v.loewis.de Tue Apr 30 07:28:17 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 30 Apr 2002 08:28:17 +0200 Subject: [Python-Dev] iterzip() In-Reply-To: <200204292107.g3TL7NA05691@pcp742651pcs.reston01.va.comcast.net> References: <200204292107.g3TL7NA05691@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > Should we worry about programs that don't create any cyclical garbage > for a long time, and then sudenly start creating lots of it? No. So far, there was only one bug report on SF (that I know of) where GC was not triggered early enough. In that case, the problem was that there were large strings attached to the cyclic garbage that consumed lots of memory, but contributed little to the object count. So any application that knows it creates cyclic garbage "on purpose" should explicitly invoke gc.collect from time to time to be safe. Most applications don't need to worry, as they can safely assume that users have enough spare memory for the garbage. Regards, Martin From martin@v.loewis.de Tue Apr 30 07:45:56 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 30 Apr 2002 08:45:56 +0200 Subject: [Python-Dev] iterzip() In-Reply-To: References: Message-ID: Tim Peters writes: > Half of a shadow of an idea: at least in my code, it's common to have > gazillions of tuples, and they almost always contain just strings and > numbers. Since they're immutable, they'll never contain anything else. So > they could get unlinked from cyclic gc entirely without ill effect (it's not > possible that they could ever be in a cycle). Perhaps a gen2 collection > could learn something about this and automagically untrack them. That could happen in gen0 already: when traversing an object, watch whether it contains any tracked objects. If it doesn't, invoke tp_is_immutable, if that returns true, untrack it. Regards, Martin From martin@v.loewis.de Tue Apr 30 07:50:53 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 30 Apr 2002 08:50:53 +0200 Subject: [Python-Dev] iterzip() In-Reply-To: References: Message-ID: Tim Peters writes: > > Maybe a (potentially) infinite number of generations? (Or at least a > > fixed limit that never gets reached in practice.) Wouldn't this have > > the same effect as increasing the threshold exponentially? > > Yes, but it wouldn't get the effect of decreasing the threshold > exponentially when gc suddenly starts getting lots of trash back from the > oldest generation. Only adaptive schemes actually adapt . That is no problem: if the application suddenly starts to create much new garbage, that garbage won't be in the oldest generation. You only run into this case when you release essentially all past objects. In that case, you just have to hope that the application will terminate soon, anyway, or that the next processing cycle will produce the same amount of garbage, triggering a collection in the oldest generation. Regards, Martin From martin@v.loewis.de Tue Apr 30 08:01:17 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 30 Apr 2002 09:01:17 +0200 Subject: [Python-Dev] Unicode In-Reply-To: <03B68F31-5AF4-11D6-895F-003065517236@oratrix.com> References: <03B68F31-5AF4-11D6-895F-003065517236@oratrix.com> Message-ID: Jack Jansen writes: > It could be argued that open(..., 'r').read() returns a text string > and open(..., 'rb').read() returns a binary blob. That can't work: to get a text string, you need to know the encoding. Regards, Martin From martin@v.loewis.de Tue Apr 30 07:58:37 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 30 Apr 2002 08:58:37 +0200 Subject: [Python-Dev] Built in objects supporting slices In-Reply-To: <200204282230.g3SMUM217981@pcp742651pcs.reston01.va.comcast.net> References: <3uqncusdqo1gs2tvvs7v878ki8ftsqbmt9@4ax.com> <200204281320.g3SDKTr16816@pcp742651pcs.reston01.va.comcast.net> <02042818534700.24988@arthur> <200204282230.g3SMUM217981@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > +1 on making the built-in types accept slice objects in their > __getitem__, __setitem__, __delitem__ methods. This is also the issue of http://python.org/sf/459235. Regards, Martin From andymac@bullseye.apana.org.au Tue Apr 30 11:44:40 2002 From: andymac@bullseye.apana.org.au (Andrew MacIntyre) Date: Tue, 30 Apr 2002 21:44:40 +1100 (edt) Subject: [Python-Dev] iterzip() In-Reply-To: Message-ID: On Mon, 29 Apr 2002, Tim Peters wrote: > That is, at first glance it's *much* faster to do a million appends in pure > Python than it is to let zip do them at C speed. I'm curious about what > people get for this program on other platforms (pymalloc on or off may make > a dramatic difference too -- or not, depending on how the platform malloc > works). OS/2 EMX (2.2.1 without pymalloc) justpush 0.59 justzip 8.85 OS/2 EMX (recent CVS with pymalloc) justpush 0.54 justzip 8.81 FreeBSD 4.4 (2.1.1 w/o pymalloc) justpush 89.72 justzip 110.41 FreeBSD 4.4 (recent CVS with pymalloc) justpush 19.21 justzip 46.32 The FreeBSD box is more mature hardware (P5-166). I'm surprised at the difference in the 2 sets of results on it. AFAIK, the compiler version and switches are identical for the two interpreters (the 2.1.1 is from the binary package on the FreeBSD 4.4 CDs). -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au | Snail: PO Box 370 andymac@pcug.org.au | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From guido@python.org Tue Apr 30 13:50:16 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 30 Apr 2002 08:50:16 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: Your message of "Tue, 30 Apr 2002 16:13:53 +1200." <200204300413.QAA02583@s454.cosc.canterbury.ac.nz> References: <200204300413.QAA02583@s454.cosc.canterbury.ac.nz> Message-ID: <200204301250.g3UCoGW16679@pcp742651pcs.reston01.va.comcast.net> [Guido] > > atom: '(' [testlist] ')' | '[' [listmaker] ']' | '{' [dictmaker] '}' > > | '`' testlist '`' | NAME | NUMBER | STRING+ | 'None' > > > > It needs to add 'None' as an alternative to all other places where > > currently NAME occurs. [Greg E] > Wait a minute, you're going about this all wrong! > > Instead of making None a reserved word, why not just make it > a non-reserved keyword when used as a variable name? What's that? > Then you wouldn't have to change the grammar at all, and there would > be no problem with the use of None in other contexts, such as > attribute names. Unfortunately, what's a variable in one context is an attribute in another. I don't think I *want* people to use None as an attribute name (or in any other context). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 30 13:52:51 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 30 Apr 2002 08:52:51 -0400 Subject: [Python-Dev] Billions of gc's In-Reply-To: Your message of "Tue, 30 Apr 2002 00:21:26 EDT." <20020430042126.GA19504@panix.com> References: <20020430025652.GA29745@panix.com> <20020430042126.GA19504@panix.com> Message-ID: <200204301252.g3UCqpU16752@pcp742651pcs.reston01.va.comcast.net> > Here's a question: suppose we've got a database result with 10K rows (I'd > say that is fairly common), and we're processing each row with a regex > (something that can't be done in SQL). What's a ballpark for gc overhead > before and after your fix? (I'm still not set up to compile CVS, so I > can't do it myself.) For 10K objects, the GC overhead is negligeable. Jeremy did 100K tuples and still found that 60% of the time was in malloc. You only get in trouble when you approach a million tuples. Remember, it's quadratic. That means it gets 100x worse with every multiplying factor 10 -- but also that it gets 100x better with every division by 10 (until you run into other effects, of course). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 30 13:52:57 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 30 Apr 2002 08:52:57 -0400 Subject: [Python-Dev] Billions of gc's In-Reply-To: Your message of "Mon, 29 Apr 2002 23:55:41 EDT." References: Message-ID: <200204301252.g3UCqvI16760@pcp742651pcs.reston01.va.comcast.net> > Well, millions of live objects is common but isn't a problem. The glitch > we're looking at it is surprising slowdown with millions of live *container* > objects. The latter isn't so common. But millions of tuples are not uncommon. They're probably the only thing to worry about here. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 30 13:58:12 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 30 Apr 2002 08:58:12 -0400 Subject: [Python-Dev] iterzip() In-Reply-To: Your message of "30 Apr 2002 08:45:56 +0200." References: Message-ID: <200204301258.g3UCwDU16809@pcp742651pcs.reston01.va.comcast.net> [Tim] > > Half of a shadow of an idea: at least in my code, it's common to > > have gazillions of tuples, and they almost always contain just > > strings and numbers. Since they're immutable, they'll never > > contain anything else. So they could get unlinked from cyclic gc > > entirely without ill effect (it's not possible that they could > > ever be in a cycle). Perhaps a gen2 collection could learn > > something about this and automagically untrack them. [MvL] > That could happen in gen0 already: when traversing an object, watch > whether it contains any tracked objects. If it doesn't, invoke > tp_is_immutable, if that returns true, untrack it. Unfortunately, the visit API doesn't make it easy to watch this; a tuple calls visit() on its items but learns nothing except whether it failed. (I've never seen a visit() implementation that could fail, so I'm not sure even why the return code exists.) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Apr 30 13:59:58 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 30 Apr 2002 08:59:58 -0400 Subject: [Python-Dev] Unicode In-Reply-To: Your message of "30 Apr 2002 09:01:17 +0200." References: <03B68F31-5AF4-11D6-895F-003065517236@oratrix.com> Message-ID: <200204301259.g3UCxwp16822@pcp742651pcs.reston01.va.comcast.net> > Jack Jansen writes: > > > It could be argued that open(..., 'r').read() returns a text string > > and open(..., 'rb').read() returns a binary blob. > > That can't work: to get a text string, you need to know the encoding. > > Regards, > Martin In my ideal rewrite of the I/O subsystem, an encoding is specified (or a site-or-app-specific default encoding used) when a file is opened. --Guido van Rossum (home page: http://www.python.org/~guido/) From ark@research.att.com Tue Apr 30 14:56:30 2002 From: ark@research.att.com (Andrew Koenig) Date: 30 Apr 2002 09:56:30 -0400 Subject: [Python-Dev] Making None a keyword In-Reply-To: <200204301250.g3UCoGW16679@pcp742651pcs.reston01.va.comcast.net> References: <200204300413.QAA02583@s454.cosc.canterbury.ac.nz> <200204301250.g3UCoGW16679@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido> I don't think I *want* people to use None as an attribute name Guido> (or in any other context). Nor, I think, do you want people trying to redefine it. -- Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark From martin@v.loewis.de Tue Apr 30 18:28:38 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 30 Apr 2002 19:28:38 +0200 Subject: [Python-Dev] iterzip() In-Reply-To: <200204301258.g3UCwDU16809@pcp742651pcs.reston01.va.comcast.net> References: <200204301258.g3UCwDU16809@pcp742651pcs.reston01.va.comcast.net> Message-ID: Guido van Rossum writes: > > That could happen in gen0 already: when traversing an object, watch > > whether it contains any tracked objects. If it doesn't, invoke > > tp_is_immutable, if that returns true, untrack it. > > Unfortunately, the visit API doesn't make it easy to watch this; a > tuple calls visit() on its items but learns nothing except whether it > failed. (I've never seen a visit() implementation that could fail, so > I'm not sure even why the return code exists.) However, the visit function could record this result. For example, for move_root_reachable: struct m_r_r{ PyGC_Head *reachable; int immutable_nonnested; }; static int visit_move(PyObject *op, struct m_r_r *tolist) { if(PyObject_IS_GC(op)){ if(GC_is_tracked(op)) m_r_r->tolist.immutable_nonnested = 0; ... } return 0; } static void move_root_reachable(PyGC_Head *reachable) { struct m_r_r my_m_r_r; my_m_r_r.reachable = reachable; for (; gc != reachable; gc=gc->gc.gc_next) { /* careful, reachable list is growing here */ PyObject *op = FROM_GC(gc); traverse = op->ob_type->tp_traverse; my_m_r_r.immutable_nonnested = 1; (void) traverse(op, (visitproc)visit_move, (void *)reachable); if(my_m_r_r.immutable_nonnested && op->ob_type->tp_immutable(op)){ GC_Untrack(op); } } } I'm not sure whether move_root_reachable would be the right place to untrack those objects, but the information is all there. Regards, Martin From andymac@bullseye.apana.org.au Tue Apr 30 14:46:34 2002 From: andymac@bullseye.apana.org.au (Andrew MacIntyre) Date: Wed, 1 May 2002 00:46:34 +1100 (edt) Subject: [Python-Dev] timeoutsocket patch In-Reply-To: <3CCCE4D4.C005AA7F@3captus.com> Message-ID: On Mon, 29 Apr 2002, Bernard Yue wrote: > Just found http://www.ia.amu.edu.pl/edm2/0401/socket3.html on the > web. The documentation above mentioned select() support in IBM TCP/IP > development toolkit for OS/2. So maybe OS/2 support select(). I > remember TCP/IP support for OS/2 comes as an optional package of IBM LAN > Manager client, back in 94-95. Am I correct, Andrew? Anyway, since you > have access to OS/2, can you help us to dig in a bit? I only have v4 (not v4.5 or eCS), but its still treated there as a "optional" package, certainly from the point of view of maintenance. I don't have VAC++ or a complete set of API docs either, but it wouldn't surprise me that select() is available in the socket library, exclusively for sockets. I use EMX, which has a select() that does what needs to be done to support sockets as well as files and (I think) pipes. I hope to have a look at your bits this weekend. Will let you know what transpires. -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au | Snail: PO Box 370 andymac@pcug.org.au | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From fredrik@pythonware.com Tue Apr 30 20:29:56 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Tue, 30 Apr 2002 21:29:56 +0200 Subject: [Python-Dev] Built in objects supporting slices References: <3uqncusdqo1gs2tvvs7v878ki8ftsqbmt9@4ax.com> <200204281320.g3SDKTr16816@pcp742651pcs.reston01.va.comcast.net> Message-ID: <024a01c1f07d$6a38a540$ced241d5@hagrid> guido wrote: > Yes. No PEP is needed, but I'm -0 on the idea -- I don't see what use > there is for this besides theoretical neatness; I'm sure it > complicates a lot of code. > > IOW What's the use case? the only use case I can think of is python training -- for some reason, my students always ask for this, and it's a bit annoying having to explain that "there's a syntax for this, but it doesn't work". except for that, I miss this about once every five years... From bernie@3captus.com Tue Apr 30 20:50:02 2002 From: bernie@3captus.com (Bernard Yue) Date: Tue, 30 Apr 2002 13:50:02 -0600 Subject: [Python-Dev] timeoutsocket patch References: Message-ID: <3CCEF569.DC12D0DB@3captus.com> Andrew MacIntyre wrote: > > On Mon, 29 Apr 2002, Bernard Yue wrote: > > > Just found http://www.ia.amu.edu.pl/edm2/0401/socket3.html on the > > web. The documentation above mentioned select() support in IBM TCP/IP > > development toolkit for OS/2. So maybe OS/2 support select(). I > > remember TCP/IP support for OS/2 comes as an optional package of IBM LAN > > Manager client, back in 94-95. Am I correct, Andrew? Anyway, since you > > have access to OS/2, can you help us to dig in a bit? > > I only have v4 (not v4.5 or eCS), but its still treated there as a > "optional" package, certainly from the point of view of maintenance. > So does OS/2 now support TCP/IP by default? > I don't have VAC++ or a complete set of API docs either, but it wouldn't > surprise me that select() is available in the socket library, exclusively > for sockets. > > I use EMX, which has a select() that does what needs to be done to support > sockets as well as files and (I think) pipes. > [snip] > I hope to have a look at your bits this weekend. Will let you know what > transpires. > I've just upload an alpha version of unit case for the timeout socket (http://www.3captus.com/Downloads/test_sockettimeout.py). Please give it a try. Bernie > -- > Andrew I MacIntyre "These thoughts are mine alone..." > E-mail: andymac@bullseye.apana.org.au | Snail: PO Box 370 > andymac@pcug.org.au | Belconnen ACT 2616 > Web: http://www.andymac.org/ | Australia -- There are three schools of magic. One: State a tautology, then ring the changes on its corollaries; that's philosophy. Two: Record many facts. Try to find a pattern. Then make a wrong guess at the next fact; that's science. Three: Be aware that you live in a malevolent Universe controlled by Murphy's Law, sometimes offset by Brewster's Factor; that's engineering. -- Robert A. Heinlein From ikjos@email.uophx.edu Tue Apr 30 21:05:34 2002 From: ikjos@email.uophx.edu (Ian Kjos) Date: Tue, 30 Apr 2002 15:05:34 -0500 Subject: [Python-Dev] millions of tuples Message-ID: <01bb01c1f082$63f50020$cc058dd8@avatartech.com> Two things spoken by Guido: > But millions of tuples are not uncommon. They're probably the only > thing to worry about here. > Unfortunately, the visit API doesn't make it easy to watch this[*]; a > tuple calls visit() on its items but learns nothing except whether it > failed. (I've never seen a visit() implementation that could fail, so > I'm not sure even why the return code exists.) * "this" is the number of tracked objects visited during a GC scan, as described by MvL If visit() does something sensible (like traversing a directed, potentially cyclic graph), then the only way it could fail is to not return. No doubt there is a clear record of who calls this function. Why not we change the semantics of visit() in this case to provide the required information? Make visit() return a non-negative integer for "number of tracked objects seen". If someone can find a good reason for error codes, then there is the negative half of the integer number line. (/me is going out on a limb and hypothesizing that visit() is in C for all relevant cases.) From mgilfix@eecs.tufts.edu Tue Apr 30 21:15:48 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Tue, 30 Apr 2002 16:15:48 -0400 Subject: [Python-Dev] ServerSocket patch Message-ID: <20020430161547.A4004@eecs.tufts.edu> I've just logged the small patch to ServerSocket.py into sourceforge that "fixes", or rather makes apparent the bug Bernard had discovered when writing his unit test for the socket module. The fix was simple: in keeping with the style of the module, a member variable was added to the ThreadingMixIn class 'daemon_threads' which decideds whether threads start in daemon mode. Thus to fix the server problem in the previous thread of convo, Bernard needed to do: class ThreadingTCPServer( SocketServer.ThreadingMixIn, MyTCPServer): daemon_threads = 1 for his final version of the server. Or until this patch is accepted, he could subclass off the threadingMixIn class and change the process_request function. The patch is #550765 in SF. -- Mike -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From fredrik@pythonware.com Tue Apr 30 22:22:17 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Tue, 30 Apr 2002 23:22:17 +0200 Subject: [Python-Dev] ServerSocket patch References: <20020430161547.A4004@eecs.tufts.edu> Message-ID: <039d01c1f08d$1d1b54f0$ced241d5@hagrid> Michael Gilfix wrote: > I've just logged the small patch to ServerSocket.py into sourceforge > that "fixes", or rather makes apparent the bug Bernard had discovered > when writing his unit test for the socket module. fwiw, the usual meaning of a "patch" is the output from the diff -c command, not a modified copy of the entire file... (see the python-on-sf FAQ for more info) From niemeyer@conectiva.com Tue Apr 30 22:30:53 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Tue, 30 Apr 2002 18:30:53 -0300 Subject: [Python-Dev] Re: os.environ.copy()['COPY_HUH'] = "not really" In-Reply-To: <20020430152022.I17488@unpythonic.net> References: <20020430152022.I17488@unpythonic.net> Message-ID: <20020430183052.A30307@ibook.distro.conectiva> > Under Python2.1 and earlier (back to 1.5.2): > >>> import os > >>> os.environ.copy()['COPY_HUH'] = "not really" > >>> print os.environ['COPY_HUH'] > not really > > Under even 2.3a0 (CVS): > >>> import os > >>> os.environ.copy()['COPY_HUH'] = "not really" > >>> os.system("echo $COPY_HUH") > not really > 0 Here's the problem: >>> print os.environ.__class__ os._Environ >>> copy = os.environ.copy() >>> print copy.__class__ os._Environ It's not a real dictionary, but an os._Environ class. Copying it creates instances of the same class, and that class' behavior is to insert the item in the environment. I belive this should be avoided in this case, returning a true dictionary from copy(). I'll send a patch to SourceForge, if you haven't done so yet. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From mgilfix@eecs.tufts.edu Tue Apr 30 22:33:48 2002 From: mgilfix@eecs.tufts.edu (Michael Gilfix) Date: Tue, 30 Apr 2002 17:33:48 -0400 Subject: [Python-Dev] ServerSocket patch In-Reply-To: <039d01c1f08d$1d1b54f0$ced241d5@hagrid>; from fredrik@pythonware.com on Tue, Apr 30, 2002 at 11:22:17PM +0200 References: <20020430161547.A4004@eecs.tufts.edu> <039d01c1f08d$1d1b54f0$ced241d5@hagrid> Message-ID: <20020430173348.E4004@eecs.tufts.edu> My mistake. I noticed some patches included the entire file if it was only one file and got lazy. Fixed in the SF tracker. -- Mike On Tue, Apr 30 @ 23:22, Fredrik Lundh wrote: > Michael Gilfix wrote: > > > I've just logged the small patch to ServerSocket.py into sourceforge > > that "fixes", or rather makes apparent the bug Bernard had discovered > > when writing his unit test for the socket module. > > fwiw, the usual meaning of a "patch" is the output from the > diff -c command, not a modified copy of the entire file... > > (see the python-on-sf FAQ for more info) > > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev `-> (fredrik) -- Michael Gilfix mgilfix@eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html From niemeyer@conectiva.com Tue Apr 30 22:41:52 2002 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Tue, 30 Apr 2002 18:41:52 -0300 Subject: [Python-Dev] Re: os.environ.copy()['COPY_HUH'] = "not really" In-Reply-To: <20020430183052.A30307@ibook.distro.conectiva> References: <20020430152022.I17488@unpythonic.net> <20020430183052.A30307@ibook.distro.conectiva> Message-ID: <20020430184152.B30307@ibook.distro.conectiva> > Here's the problem: [...] Sorry, the CC was for python-list, not python-dev. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] From pinard@iro.umontreal.ca Mon Apr 8 02:38:22 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 07 Apr 2002 21:38:22 -0400 Subject: [Python-Dev] Re: Stability and change In-Reply-To: <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> References: <20020406202925.GB14990@panix.com> <200204070118.g371IjN29027@pcp742651pcs.reston01.va.comcast.net> <02040710094202.05787@arthur> <200204072208.g37M81t08653@pcp742651pcs.reston01.va.comcast.net> Message-ID: [Guido van Rossum] > I suspect that the people who complain that Python's rate of change > is too fast are misguided. [...] So maybe there's only (or mostly) > a problem of *perception*? I use Python on a few systems and flavours, and Python versions are not the same on all machines. At places, people ask me to limit myself to 1.5.2. At home, I have 2.2, but 2.0 is currently the common denominator for the set of machines on which I do my own projects (I physically travel between sites). For one, I have no real problem aiming 1.5.2 or 2.0 instead of 2.2, when projects or teams require it. Despite I much enjoy most of the new features, I still have to be careful about various development contexts. I would presume that for a newcomer to Python having to ping and pong between different Python versions, not firmly aware of the differences, might feel irritated by things working here, but not there. This is just an hypothesis, as I did not experience such an irritation myself: because when one knows which version has which features, there is no real problem. I guess the difficulty comes from the time evolution takes for new versions of Python to get installed universally, so new features are just there, dependably, blindly, everywhere. This means a few years at least. The difficulty does _not_ come from forward incompatibilities (there were a few around sockets, say, but these were simple things, not really worth complaining about in my opinion), in my experience, forward migration has been either very easy or just fully transparent. The current pace of change does not look unreasonable to me. I wonder if the perception would not improve, if there was a bit more documentation or publicity about what users should do about version inter-operability. If users understand what they choose, they will less likely feel changes as imposed to them. Maybe! :-) -- François Pinard http://www.iro.umontreal.ca/~pinard